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(57) In a computer processing apparatus (2), a 3D 
computer model of an object (210) is generated by 
calculating the intersections of polyhedra (520), each 
of which defines a volume of 3D space containing at 
least part of the object. The 3D points of intersection 
of the planar faces of the polyhedra are calculated and 
each 3D point is labelled with the planar faces which 
meet thereat. Tliereafter the points are connected to 
form a polygon mesh using the labels to determine which 
points should be connected together. 



Fig. 10. 
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Description 



KUnt^ti^" iT^h''*^ described in "A Volumetric Intersection Algorithm for 3d-Reconstruction Using a Boundary- 
nrl«^ .nJ^?r ^9^"^'- '"eff-cient in temis of processing resources and processing time The inventors "n the 
S^t!! ""'"^i!?" ^"^"^ '^^'^^ particular reason for this problem is thi incrernentlTroDroach o1 

intereecting a silhouette cone with a representation to generate a refined represemation (^^mS? connected 3 
foooll Ta^^^^^^^^^^ the refined representation vv^th anc^her silhouettrconTand so on 

rronhl P J^/^no.^ °' Determining the Intersection of Twj Simple Polyhedra" by M. Szilvas-Nagy in Computer 
Jf.?Th 3 (1984) pages 219-225 discloses a method for calculating the intersection of t4 sSe SSl? 
n^K^f^.^ ^^'f^f^'T ^"^ f e"^""' "^'"9 ^ "plane-sweep algorithm" to find all intersections of the SgSof S 
polyhedron with the faces of the other. While this method is useful for two polyhedra, each having a srSlI numblf of 

fhrproblem?^ '^'^^^"^ ''^^^ ""^"^^ "^^^ ""^^^ ^"^ ^'""^ *° ^^^^^^s one or more of 

nTJJLino ^'='^"^'"9 *? "'^ P''«sent invention, there is provided a computer processing method and apparatus for 
generating a non-voxel representation of an object by intersecting at least three polyhedra The pdnts 0 
rnfrff P'«"^^ ♦a<=es making up the polyhedra are calculated and tested to determine v^ich poinS 

correspond to object points. Each object point is labelled with the planar faces which meet thereat ThereX? 
tt tebels ' bet^^een the Jbject points are specified inS?nde!lcL upon 

SSJpLinn HiUfrT/i ^ calculating all of the object points before determining the connections therebetween, the 
processing to detemiine the connections needs to be performed only once, with consequential savings in processina 

mer.Tr.'t^"'^^'.*^- " w""^^." P°"""''^ '^^^"'"9 ^'^''^ °t,ject point based on the plaSr faces Sh 
ntersectto generate the point, and calculating which object points should be connected together using their Tabels 

n^lrlfin *? *® P'^s^"* invention, there is also provided a computer processing method and apparatus for 

K h.^ nZ°rtT- '^^'^'fTT °* ^" intersecting a plurality of polyhedfa. An initial 3D voSime in 

^♦o-^ ! 1°'''®''' ''^^ 'f u"^"""®^ P^"^^' ^"^ part tested against the polyhedra. If a volume oaii is 
outs.de at least one polyhedron, it is discarded because it cannot contain points representing points on^he^ct 



surface, otherwise the polyhedra surfaces intersecting the volume are processed to calculate the points of 
intersection thereof. A 3D computer model comprising the calculated points of intersection is therefore generated. 
[0013] In this \Aay, all intersections of the polyhedra surfaces do not need to be calculated, because the 
processing prevents intersections which cannot represent points on the object from being calculated, resulting in 
savings of processing resources and processing time. 

[0014] Preferably, the processing is carried out in an even more efficient way by repeatedly subdividing the 
initial volume, and calculating the intersections of polyhedra surfaces for a volume part only if the volume part Is 
Intersected by a small number of polyhedra surfaces. By way of explanation, in an embodiment, each Intersection 
comprises the Intersection of three polyhedra surfaces. Therefore, each respective combination of three surfaces 
which intersect a volume part must be tested to determine If they Intersect, resulting In a considerable number of 
calculations If a volume part is Intersected by a large number of polyhedra surfaces, with many of the calculations 
being wasted because the point of intersection does not represent a point on the object. However, by repeatedly sub- 
dividing the initial volume, large amounts of the volume can be discarded without having to calculate any 
intersections, and by calculating intersections only when a sufficiently small volume part has been obtained that it 
is intersected by a small number of polyhedra surfaces, the number of intersections calculations which actually need 
to be performed are minimised. 

[0015] The present invention also provides a computer program product, embodied for example as a storage 
device or signal, including instructions for causing a programmable processing apparatus to become operable to 
perform a method as set out above or to become configured as an apparatus as set out above. 

[0016] Embodiments of the Invention will now be described, by way of example only, with reference to the 
accompanying drawings in which: 

Figure 1 schematically shows the components of a first embodiment of the Invention, together with the notional 
functional processing units Into which the processing apparatus component may be thought of as being configured 
when programmed by programming Instructions; 

Figure 2 illustrates the recording of Images of an object for v\lilch a 3D computer mode! Is to be generated; 

Figure 3 illustrates Images of the object which are input to the processing apparatus In Figure 1 ; 

Figure 4, comprising Figures 4a and 4b, shows the processing operations performed by the processing apparatus 
in Figure 1 to process Input data; 

Figure 5, comprising Figures 5a, 5b and 5c shows the processing operations perfomaed at step S4-8 in Figure 4; 
Figure 6 shows the processing operations perfomned at step S4-12 in Figure 4; 

Figure 7 Illustrates an example of the display on the display device of Figure 1 during processing at steps S6-2 
and S6-4 In Figure 6; 

Figure 8 shows the processing operations performed at step S4-16 in Figure 4; 

Figure 9 shows an example of a subject object silhouette approximated with straight lines as a result of the 
processing perfomied at step S8-2; 

Figure 10 shows an example of a polyhedron defined by a silhouette, and illustrates how polygons from different 
polyhedra intersect to generate points in the 3D computer model; 

Figure 11, comprising Figures 1 la and 11b, shows the processing operations performed at step S8-4 in Figure 8; 
Figure 12 shows an example Illustrating the processing performed at steps S11-4 to S11-8 in Figure 1 1 ; 
Figure 13 shows the processing operations perfonned at step S8-6 in Figure 8; 

Figures 14a, 14b and 14c show an example Illustrating howa 3D volume Is defined at step S13-2 In Figure 13; 

Figure 15, comprising Figures 15a, 15b, 15c and 15d, shows the processing operations performed at step S13-8 in 

Figure 13; 

Figure 16 shows an example illustrating the projection of a volume into a 2D image at step Si 5-2 in Figure 15; 

Figures I7a-h illustrate eight respective relationships between the projection of a volume in a tv«D-dimensional 
image and the edges of the object silhouette In the Image, processing being carried out In the first embodiment 
to test which of these relationships exists; 



Rgure 18 shows the processing operations performed at step S15-16 in Figure 15; 

Figure 19 shows an example illustrating the processing perfomied at steps S18-2, S18-4 and S18-6 in Figure 18; 

Figure 20 shows the processing operations performed at step S15-18 in Figure 15; 
• Figure 21 shows the processing operations performed at step S16-38 in Figure 15; 

Rgure 22 shows the processing operations performed at step S13-12 in Figure 13; 

Figure 23 shows the processing operations performed at step S8-8 in Figure 8; 

Rgure 24 shows an example to illustrate the processing performed at step S23-2 in Figure 23; 

Figure 25 shows the processing operations performed at step S23-2 in Figure 23; 

Rgure 26 shows the processing operations performed at step S23-4 in Figure 23; 

Rgure 27 shows an example to illustrate the processing perfomied at step S23-6 in Figure 23; 

Rgure 28 shows another example to illustrate the processing performed at step S23-6 in Figure 23; 
Rgure 29 shows a further example to illustrate the processing performed at step S23-6 in Figure 23; 
R^e 23°' ^'^""'^^ 30a. 30b. 30c and 30d. shows the processing operations performed at step S23-6 in 

f^^^lnnV, s'=^«'"a<'<»"y stio^ the components of a second embodiment of the invention, together with the notional 
b^, ^^I^^^^Si^:^'^ --P°-nt may be thought^of as beinTc^S 

Rgure 32 shows the processing operations perfomied by the processing apparatus in Rgure 31 to process input 
Rgure 33a shows an example of component objects generated at step S32-2 in Rgure 32; 
FiSesIf ^"""^ ^ ^"^""^'^ ^""^'^^ component objects from Figure 33a are positioned at step S32-4 in 

Figure 34 shows the processing operations performed at step S32-6 in Figure 32; 
Figure 35 illustrates howa bounding volume is defined at step S34-2.in Figure 34; 
Rgure 36 illustrates how the volume defined at step S34-2 is subdivided at step S34-4 in Figure 34; 
SS^RgSfiT ^ '=0"*aining a hole representing a component object is generated at step 

FT?u"r?s2?' ""'S"'® and 38c. shows the processing operations performed at step S34-10 in 

Figure 39 shows the processing operations perfomied at step S38-18 in Figure 38; 

and sllt°in F^^ufe 39?°"" ^ "'^ ^^""^'^ *° '""^'^'^ processing operations perfomied at steps S39-2. S39-4 
s'^ITiid SslSi^ RgSe 39! ^'^"""^ '° processing operations performed at steps S39-2. 

Figure 42 shows the processing operations perfomied at step S34-14 in Figure 34; 
Rgure 43 shows an example to illustrate the result of the processing performed at step S32-6 in Figure 32; 
Rgure 44 shows the processing operations perfomied in the second embodiment at step S23-2 in Rgure 23; and 



Figure 45 shows the processing operations performed in a third embodiment at step S4-16 in Figure 4. 



First Embodiment 

[0017] Referring to Figure 1, an embodiment of the invention comprises a processing apparatus 2, such as a 
personal computer, containing, in a conventional manner, one or more processors, memories, graphics cards etc, 
together with a display device 4, such as a conventional personal computer monitor, user input devices 6, such as a 
keyboard, mouse etc, a printer 8, and a display panel 10 comprising a flat panel having controllable pixels, such as 
the PL400 manufactured by WACOM. 

[0018] The processing apparatus 2 is programmed to operate in accordance with programming instructions input, 
for example, as data stored on a data storage medium, such as disk 12, and/or as a signal 14 input to the processing 
apparatus 2, for example from a remote database, by transmission over a communication network (not shown) such as 
the Internet or by transmission through the atmosphere, and/or entered by a user via a user input device 6 such as a 

keyboard. 

[0019] As will be described in more detail below, the programming instructions comprise instructions to cause 
the processing apparatus 2 to become configured to process input data defining a plurality of images of one or more 
subject objects recorded at different positions and orientations to calculate the positions and orientations at 
which the input images were recorded, and to use the calculated positions and orientations to generate data defining 
a three-dimensional computer model of the subject object(s). In this embodiment, the subject object(s) is imaged on a 
calibration object (a two-dimensional photographic mat in this embodiment) which has a known pattern of features 
thereon, and the positions and orientations at which the input images were recorded are calculated by detecting the 
positions of the features of the calibration object pattern in the images. The 3D computer model of the subject 
object(s) is generated using a novel and inventive polyhedra intersection technique, in which the outline of the 
subject object(s) in each image is divided into parts and processing is performed to calculate 3D points 
representing points on the surface of the subject object by determining the intersections of the polyhedra defined 
by the 3D projections of the outline parts. As will be described in detail below, this processing Is performed in a 
particularly efficient way, In particular because it avoids calculating Intersections of faces within the polyhedra 
which cannot result in 3D points on the subject object surface. 

[0020] When programmed by the programming Instructions, processing apparatus 2 can be thought of as being 
configured as a number of functional units for performing processing operations. Examples of such functional units 
and their Interconnections are shown In Figure 1. The units and Interconnections illustrated in Figure 1 are, however, 
notional and are shown for Illustration purposes only to assist understanding; they do not necessarily represent 
units and connections into which the processor, memory etc of the processing apparatus 2 become configured. 
[0021] Referring to the functional units shown in Figure 1, a central controller 20 Is arranged to process 
inputs from the user input devices 6, and also to provide control and processing for the other functional units. 
Memory 24 is provided for use by central controller 20 and the other functional units. 

[0022] Mat generator 30 is arranged to generate control signals to control printer 8 or display panel 10 to 
print a calibration pattern on a recording medium such as a piece of paper to form a printed photographic mat, or to 
display the calibration pattern on display panel 10 to display a photographic mat. As will be described in more 
detail below, the photographic mat comprises a predetermined calibration pattern of features and the object(s) for 
which a three-dimensional computer model is to be generated Is placed on the printed photographic mat 34 or on the 
display panel 10 on which the calibration pattern is displayed. Images of the object and the calibration pattern are 
then recorded and input to the processing apparatus 2. Mat generator 30 is arranged to store data defining the 
calibration pattern of features printed or displayed on the photographic mat for use by the processing apparatus 2 
in calculating the positions and orientations at which the Input Images were recorded. More particularly, in this 
embodiment, mat generator 30 Is arranged to store data defining the pattern of features together with a coordinate 
system relative to the pattern of features (which, In effect, defines a reference position and orientation of the 
calibration pattern), and processing apparatus 2 is arranged to calculate the positions and orientations at which 
the input images were recorded in the defined coordinate system (and thus relative to the reference position and 
orientation). 

[0023] In this embodiment, the calibration pattern on the photographic mat comprises spatial clusters of 
features for example as described in PCT application GBOO/04469 (the full contents of which are incorporated herein 
by cross-reference) or any known pattem of features, such as a pattem of coloured dots, with each dot having a 
different hue/brightness combination so that each respective dot is unique, for example as described in JP-A-9-170914, 
a pattern of concentric circles connected by radial line segments with known dimensions and position markers in each 
quadrant, for example as described in "Automatic Reconstruction of 3D Objects Using A Mobile Camera" by Niem in 
Image and Vision Computing 17 (1999) pages 125-134, or a pattern comprising concentric rings with different 
diameters, for example as described in "The Lumigraph" by Gortler et al in Computer Graphics Proceedings, Annual 
Conference Series, 1996 ACM-0-8979 1-764-4/96/008. 

[0024] In the remainder of the description, It will be assumed that the calibration pattern is printed by 
printer 8 on a recording medium (in this embodiment, a sheet of paper) to generate a printed photographic mat 34, 
although, as mentioned above, the calibration pattern could be displayed on display panel 10 Instead. 
[0025] Input data store 40 is arranged to store input data input to the processing apparatus 2 for example as 
data stored on a storage device, such as disk 42, as a signal 44 transmitted to the processing apparatus 2, or using 



a user input device 6. The input data defines a plurality of images of one or more subject objects on the photographic 
mat recorded at different positions and orientations, and an input image showing the background against which the 
object(s) was imaged together with part of the photographic mat to show the background colour thereof or a different 
object having the same colour as the background colour of the mat. In addition, in this embodiment, the input data 
also includes data defining the intrinsic parameters of the camera which recorded the images, that is the aspect 
ratio, focal length, principal point (the point at which the optical axis intersects the imaging plane)' first order 
radial distortion coefficient, and skew angle (the angle between the axes of the pixel grid; because the axes may 
not be exactly orthogonal). 

[0M6] The input data defining the input images may be generated for example by downloading pixel data from a 
digital camera which recorded the images, or by scanning photographs using a scanner (not shown). The input data 
defining the intrinsic camera parameters may be input by a user using a user input device 6. 

[0027] C^amera calculator 50 is arranged to process each input image to detect the positions in the image of the 
features in the calibration pattern of the photographic mat and to calculate the position and orientation of the camera 
when the input image was recorded. 

[0028] Image data segmenter 60 is arranged to process each input image to separate image data corresponding to 
the subject object from other image data in the image. y 
[0029] Image segmentation editor 70 is operable, under user control, to edit the segmented image data generated 
by image data segmenter 60. As naaII be explained in more detail below, this allows the user to correct an image 
segmentation produced by image data segmenter 60, and in particular for example to correct pixels mistakenly 
determined by image data segmenter 60 to relate to the subject object 210 (for example pixels relating to marks or 
nvfk r^^^^^^^^ ^ ^♦u'^^^'u °" "^'^.^ photographic mat 34 and subject object are placed for imaging, 

PD^els relating to shadows on the photographic mat 34 and/or surface on which it is placed, and pixels relating to a 
^wo'tifh I"'® 2"**^® photographic mat 34 which touches the outline of the subject object in the input image 
le on®" . ° 7!is*^'<®"'y classified during image data segmentation and to lead to inaccuracies in the 
resulting 3D computer model if not corrected). 

[0030] Surface modeller 80 is arranged to process the segmented image data produced by image data segmenter 
^t^^l'r^^^ segmentation editor 70 and the data defining the positions and orientations at vN^ich the images were 
LnrfJ^fe K ^A'^^T'^ calculator 50. to generate data defining a 3D computer model representing the actual 

surfaces of the object(s) in the input images. 

[0031] Surface modeller 80 comprises a silhouette approximator 82, a 3D point calculator 84, and a polygon 
generator 86- 

[0032] Silhouette approximator 82 is arranged to process the segmented image data generated by image data 

oh?^Tt!c !n tLT,lLT^^^^T^^2^r'°'' ^° ^^^^ '"P"* ™«9« to approximate the silhouette of the subject 
ODject(s) in each image with straight line segments. 

[0033] 3D point calculator 84 is arranged to process the straight line segments generated by silhouette 
dfrnT^'S cnf.'° intersections of polygons defined by the projection of *e line segments into r?hree 

dimensional space. TTiese intersections define 3D points which are potentially on the surface of the subject object(s) 

in'theTrfafe obSs) " '° '° *° "^'^ r°ep!£en7S?poS 

[0034] Polygon gerierator 86 is arranged to connect the 3D points generated by 3D point calculator 84 to form a 
polygon mesh representing the surface of the subject object(s). k-ou" odiouiaior 04 ro lorm a 

f^^t? «n '^li'' m'^''!?'' u*® '"tersections of the polygons calculated by 3D point calculator 84 define 3D 

S^niLo?86 JlS,h"Jl?Il''th'"'''^^^^ ""^rif^- ^'1^ ^'^cordingly these 3D points are connected by pSfygon 
fuKoh£«^> P«/th-^ ° the verfices of the polygons in the polygon mesh representing the surface of the 
t^^ai lLveSs ' 3D points calculated by 3D point calculator 84 will be referred 

?Se r.J^ZlTX'suS:!e'^^^^^^ '^'^"^^ '"-^"^ '^^^ -^-'"9 onto the 

[0037] Display processor 110, under the control of central controller 20, is arranged to disolav imaaes and 
instructions to a user via display device 4 during processing to generate the 3^ computlrmSe? In addSn under 
oAuTlt^r''^' controller 20 display processor 110 is also arranged to display images oTthe 3D co^^^^^^^ 
of the object from a user-selected viewpoint by processing the surface model data generated by surface modelter 80 
fnnooT''^'''"^*®'^"'^ '^^^ produced by surface texturer 90 onto the surface model ^° 

*r,g i;r2'a";is'" •» "-^^ • - 

LTo'fo, 'TT' M i» Pte»«l on ' surtace 200, and lha subiect Mad 



appears on the surface 200 (and hence in input images), processing can be performed In this embodiment to 
accommodate this by allowing the user to edit segmentation data produced by image data segmenter 60, as will be 
described in more detail below. 

[0041] Images of the object 210 and photographic mat 34 are recorded at different positions and orientations to 
show different parts of object 210 using a digital camera 230. In this embodiment, data defining the images recorded 
by camera 230 is input to processing apparatus 2 as a signal 44 along wire 232. 

[0042] More particularly, in this embodiment, camera 230 remains in a fixed position and photographic mat 34 
^ with object 210 thereon is moved (translated) and rotated (for example in the direction of arrow 240) on surface 200, 
and photographs of the object 210 at different positions and orientations relative to the camera 230 are recorded. 
During the rotation and translation of the photographic mat 34 on surface 200, the object 210 does not move relative 
to the mat 34. 

[0043] Figure 3 shows examples of images 300, 302, 304 and 306 defined in data input to processing apparatus 2 
^0 of the object 210 and photographic mat 34 in different positions and orientations relative to camera 230. 

[0044] In this embodiment, following the recording and input of images of object 210 and photographic mat 34, a 
further image is recorded and input to processing apparatus 2. This further image comprises a "background image", 
which is an image of the surface 200 and an object having the same colour as the paper on which photographic mat 34 
is printed. Such a background image may be recorded by placing a blank sheet of paper having the same colour as the 
sheet on which photographic mat 34 is recorded on surface 200, or by tuming the photographic mat 34 over on surface 
^5 200 so that the pattern thereon is not visible in the image. 

[0045] Figure 4 shows the processing operations performed by processing apparatus 2 to process input data In 
this embodiment. 

[0046] Referring to Figure 4, at step S4-2, central controller 20 causes display processor 110 to display a 
message on display device 4 requesting the user to input data for processing. 
2Q [0047] At step S4-4, data input by the user in response to the request at step S4-2 is stored in the input data 

store 40. More particularly, in this embodiment, the input data comprises image data defining the images of the 
object 210 and photographic mat 34 recorded at different positions and orientations relative to the camera 230, the 
"background Image" showing the surface 200 on which photographic mat 34 was placed to record the Input images 
together with an object having the same colour as the recording material on which the calibration pattern of 
photographic mat 34 is printed, and data defining the intrinsic parameters of the camera 230 which recorded the 
25 Input images, that is the aspect ratio, focal length, principal point (the point at which the optical axis 
intersects the imaging plane), the first order radial distortion coefficient, and the skew angle (the angle between 
the axes of the pixel grid). 

[0048] At step S4-6, camera calculator 50 processes the input data stored at step S4-4 to determine the position 
and orientation of the camera 230 relative to the calibration pattern on the photographic mat 34 (and hence relative 
to the object 210) for each input Image. This processing comprises, for each input Image, detecting the features in 
the image which make up the calibration pattern on the photographic mat 34 and comparing the features to the stored 
pattern for the photographic mat to determine the position and orientation of the camera 230 relative to the mat. 
The processing perfomned by camera calculator 50 at step S4-6 depends upon the calibration pattern of features used 
on the photographic mat 34. Accordingly, suitable processing is described, for example, in co-pending PCT 
application GBOO/04469, JP-A-9-170914, "Automatic Reconstruction of 3D Objects Using A Mobile Camera" by Niem in 
35 Image and Vision Computing 17 (1999) pages 125-134 and "The Lumigraph" by Gortler et al in Computer Graphics 
Proceedings, Annual Conference Series, 1996 ACM-0-89791 -764-4/96/008. 

[0049] At step S4-8, image data segmenter 60 processes each input Image to segment Image data representing 
the object 210 form image data representing the photographic mat 34 and the surface 200 on which the mat 34 is 
placed (step S4-8 being a preliminary step in this embodiment to generate data for use In the subsequent generation 
of a 3D computer model of the surface of object 210, as will be described in more detail below). 
40 [0050] Figure 5 shows the processing operations performed by Image data segmenter 60 at step S4-8. 

[0051] Referring to Figure 5, at steps S5-2 to 85-10, image data segmenter 60 builds a hash table of quantised 
values representing the colours in the input images which represent the photographic mat 34 and the background 200 
but not the object 210 itself. 

[0052] More particularly, at step 85-2, image data segmenter 60 reads the RBG data values for the next pixel in 
the "background image" stored at step 84-4 in Figure 4 (that is, the final image to be Input to processing apparatus 
2 which shows the surface 200 and an object having the same colour as the material on vN^ich photographic mat 34 is 

printed). 

[0053] At step 85-4, image data segmenter 60 calculates a quantised red (R) value, a quantised green (G) and a 
quantised blue (B) value for the pixel in accordance with the following equation: 

Q= (1) 

where: 

"q" is the quantised value; 
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"p" Is the R, G or B value read at step 85-2; 
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J'Jl! threshold value determining how near RGB values from an Input image showing the object 210 need to be 
to background colours to be labelled as background. In this embodiment, Is set to 4. 

fm?a\pleCS^^^^^ ^^"^^"^^ ^-"^-^ ^ ^ -'-^ -'-lated at step S5-4 

SSted ^TSithr^^^^^^ 60 applies a hashing function to the quantised R, G and B values 

calculated at step S5-4 to define a bin in a hash table, and adds the "triple" value defined at steo S5-6 to the 

t^Se^Su^ntlJ^^^^ ?^^«nHR''i'" '"^age data segmenter 6o' applies the fol^^^^^^^^ 

to the quantised R,G and B values to define the bin in the hash table: . ~ ■">« >u>n."on 

h{q) = (q,^r,^2-6+ (q„^M*2^^{q„u.&^ (2) 

[Opsq TTiat is, the bin in the hash table is defined by the three least significant bits of each colour This 
fur^ction ,s chosen to try and spread out the data into the available bins in the hash table, so S5 ea?h bin haJ 
only a small number of "triple" values. In this embodiment, at step S5-8, the "triple" value is added to the Wn 
or|^, rf it does not already exist therein, so that each "triple" value is added ^ 

fman? <;t»n= o"? t^?;. '""^^^ data Segmenter 60 detemiines whether there is another pixel' In the background 
Sbed above t l'?JuH nl^T^ ^'''i'" "background" image has been processed in the manner 

ctlours1n^h1Tac5ou'nd"?mag^ " ^"^'"'"^ 

^^nilnlTo ntl^- *°S5-48, image data segmenter 60 considers each input image in turn and uses the hash 
table to segment the data in the input image relating to the photographic mat 34 and background from the data in the 

^ So5^''^n thi?.m»iH ST is displayed'on display devL 4 ' 

SJ>c Lt JZl^ embodiment, the "background" image processed at steps S5-2 to S5-10 to generate the hash table 
ges not sh°w the calibration features on the photographic mat 34. Accordingly, the segmentation perfomied at step? 
55-12 to S5-48 does not distinguish pixel data relating to the object 210 from pixel data relating to a calibration 
Lene L°"th2'3D S^^^^ this embodiment, the processing pe'rformed by sCrfaL m^deHe? sTto 

generate the 3D computer model of the surface of object 210 is carried out in such a way that pixels relatlno to a 
SSKovJ °" photographic mat 34 do not contribute to the surface model, as ^^ll be desTribId rmore 

L?.?P? I S5-12, image data segmenter 60 considers the next input image, and at step S5-14 reads the R, G 

fnnci, "e?' *® '"P"* '""^9^ (^'^'^ ''^'"9 P«el the first time step S5-14 is performed). 

ni^ntLn R I ? 1 ' "^'^ segmenter 60 calculates a quantised R value, a quantised G va ue and a 

quantised B value for the pixel using equation (1) above. 

le'Sto a 'tri^l vJSe^-^^^' '""^^^ segmenter 60 combines the quantised R. G and B values calculated at step S5- 
[0063] At step S5-20, image data segmenter 60 applies a hashing function in accordance with equation (2) above 
i^^^!,*^"^" ^^''if ^ calculated at step S5-16 to define a bin in the hash table generated at steps S5-2 to 35-10 
in P ^^'^^^ '""^^ segmenter 60 reads the "triple" values in the hash table bin defined at step S5- 

^ SrfaSrioo"' ^ representing the colours of the material of the photographic mat 34 and the background 

, . ^1 ^'^P '"^^ segmenter 60 detemiines whether the "triple" value generated at step S5-18 of 

the pixel in the input image currently being considered is the same as any of the background "triple" values in the 
hash table bin. 

l?^^}. , determined at step S5-24 that the "triple" value of the pixel is the same as a background 

. ^.^ ^* determined that the pixel Is a background pixel and the value of the pixel is set 

to black . 

[0067] On the other hand, if it is determined at step S5-24 that the "triple" value of the pixel is not the same 
as any triple' value of the background, then, at step S5-28, it is determined that the pixel is part of the object 
210 and image data segmenter 60 sets the value of the pixel to "white". 
45 '"^^9^ segmenter 60 determines whether there is another pixel in the input image. 

Steps S5-14 to S5-30 are repeated until each pixel in the input image has been processed in the manner described 
aDove. 

[0069] At steps 85-32 to S5-46, image data segmenter 60 performs processing to correct any errors In the 
classification of image pixels as background pixels or object pixels, and to display the result. of the segmentation 
processing on display device 4. . 
so [0070] More particularly, at step S5-32. image data segmenter 60 defines a circular mask for use as a median 
filter. In this embodiment, the circular mask has a radius of 4 pixels. 

[0071] At step 85^4. Image data segmenter 60 performs processing to place the centre of the mask defined at 
step S5-32 at the centre of the next pixel In the binary image generated at steps 85-26 and 85-28 (this beinq the 
first pixel the first time step 85-34 is performed). v » 

[0072] At step 85-36. image data segmenter 60 counts the number of black pixels and the number of white oixels 
witntn the mask. . 

[0073] At step 85-38, image data segmenter 60 determines whether the number of white pixels within the mask is 



greater than or equal to the number of black pixels within the mask. 

[0074] If it is determined at step S5-38 that the number of white pixels is greater than or equal to the number 
of black pixels, then, at step S5-40 image data segmenter 60 sets the value of the pixel on which the mask is 
centred to white. On the other hand, If it is determined at step S5-38 that the number of black pixels is greater 
than the number of white pixels then, at step S5-42, image data segmenter 60 sets the value of the pixel on which 
the mask is centred to black. 

[0075] At step S5-44, display processor 110 displays on display device 4 a reduced resolution version (a "thumb 
^ nail" image) of the input image currently being processed and displays in the thumb nail image the result of the 
segmentation processing previously performed on the pixel at steps S5-34 to S5-42. Thus, the thumb nail image is 
changed incrementally to show the result of the segmentation processing as each pixel in the input image is 
processed. In this embodiment, display processor 1 10 displays the result of the segmentation processing in the thumb 
nail image so that image data in the input image which is determined to represent the background is presented as a 
10 predetermined colour, for example blue, in the thumb nail image. 

[0076] As a result of displaying the thumb nail image in this way, the user sees which parts of the input image 
have been processed and the result of the segmentation processing, and hence can determine whether any 
amendment is necessary. 

[0077] At step S5-46, image data segmenter 60 determines whether there is another pixel in the binary image, and 
steps S5-34 to S5-46 are repeated until each pixel has been processed in the manner described above. 
[0078] At step S5-48, image data segmenter 60 determines whether there is another input image to be processed. 
Steps S5-12 to S5-48 are repeated until each input image has been processed in the manner described above. 
[0079] Referring again to Figure 4, at step S4-10, central controller 20 determines whether a signal has been 
received from a user via a user input device 6 indicating that the user wishes to amend an image segmentation 
generated at step S4-8 (this signal being generated by the user in this embodiment by pointing and clicking on the 
2Q thumb nail image displayed on display device 4 corresponding to the segmentation which it is desired to amend). 

[0080] If it is determined at step S4-10 that an image segmentation is to be changed then, at step S4-12, image 
segmentation editor 70 amends the segmentation selected by the user at step S4-10 in accordance with user input 
instructions. 

[0081] Figure 6 shows the processing operations performed by image segmentation editor 70 during the interactive 
amendment of an image segmentation at step S4-12. 
25 [0082] Referring to Figure 6, at step S6-2, image segmentation editor 70 causes display processor 1 10 to display 
the image segmentation selected by the user at step S4-10 (by pointing and clicking on the corresponding thumb nail 
image) on display device 4 for editing. More particularly, referring the Figure 7, in this embodiment, the image 
segmentation 318 selected by the user at step S4-10 from all of the displayed thumb nail images 310-324 is displayed 
in a window 400 in a form larger than that in the thumb nail image. In this embodiment, the image segmentation 
displayed in window 400 has the same number of pixels as the input image which was processed to generate the 
segmentation. In addition, the border of the thumb nail image selected by the user (thumb nail image 318 in the 
example of Figure 7) is highlighted, or the thumb nail image is otherwise distinguished from the other thumb nail 
images, to Indicate that this is the segmentation displayed in enlarged form for editing. 
[0083] Also at step S6-2, image segmentation editor 70 causes display processor 110 to display a window 402 
moveable by the user over the displayed image segmentation within window 400. In addition, image segmentation editor 
35 70 causes display processor 110 to display a further window 410 in which the part of the image segmentation 
contained in window 402 is shown in magnified form so that the user can see which pixels were determined .by the 
image data segmenter 60 at step S4-8 to belong to the object 210 or to features on the photographic mat 34 and which 
pixels were determined to be background pixels. 

[0084] At step S6-4, image segmentation editor 70 changes the pixels displayed in window 410 from background 
pixels to object pixels (that is, pixels representing object 210 or features on the photographic mat 34) and/or 
changes object pixels to background pixels in accordance with user instructions. More particularly, for editing 
purposes, image segmentation editor 70 causes display processor 110 to display a pointer 412 which, in this 
embodiment, has the form of a brush, which the user can move using a user input device 6 such as a mouse to 
designate pixels to be changed in window 410. In this embodiment, each pixel which the user touches with the pointer 
412 changes to an object pixel if it v\as previously a background pixel or changes to a background pixel if it was 
previously an object pixfel. In this embodiment, the segmentation editor 70 causes display processor 110 to display a 
user-selectable button 350, the selection of which causes pointer 412 to become wider (so that more pixels can be 
designated at the same time thereby enabling large areas in window 410 to be changed quickly) and a user-selectable 
button 360, the selection of which causes the pointer 412 to become narrower. 

[0085] By performing processing in this way, the user is, for example, able to edit a segmentation generated by 
image data segmenter 60 to designate as background pixels any pixels mistakenly determined by image data segmenter 
SO 60 to relate to the subject object 210 (for example pixel data relating to the mark 220 on surface 200 which would 
not be separated from image data relating to subject object 210 by image data segmenter 60 if it has the same colour 
as a colour in subject object 210) and/or to designate as background pixels relating to each calibration feature on 
the photographic mat 34 which touches the outline of the subject object 210 in an image segmentation (as shown in 
the example of Figure 7) which, if not corrected, have been found to cause errors in the three-dimensional computer 
model of the subject object subsequently generated by surface modeller 80. Similarly, the user is able to designate 
as background pixels relating to shadows on the photographic mat 34 and/or surface 200 which have mistakenly been 
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t^'^'^^^ ^1 ''"^^^ segmenter 60 to be pixels relating to the subject object 210 

rlirkino nn « ^if^' ^® '"^^ Segmentation currently displayed (by pointing and 

dSL„^n^orP«S. |^n h"'"'' "t" ^^°-324 or by pointing and clicking on the "start processing button 34^) 

•SS4^T^*.^h 12? ''L^P'^y®'^ ^"""""^ '■'^^9« corresponding to the segmentation edited by^he user ai 

SfuSrtt^Sep S6 4 °* *° ^"^^^^'^ ^^^"9^= *° segmentation madJ by 

I0OT7] Referring again to Figure 4, at step S4-14, image segmentation editor 70 determines whether the user 
tfu^rfa^^lr^^^^^^^^^ *° " '"^^^^ ^^^^^^ POiSTnTcJikSS oTa^ 

«?l!!fLt=f«r']rj"^ determined at step S4-10 or step S4-14 that no further changes are to be made to an imaae 

" fSe sukce o'i's'Sct'oSct '° *° ^ a 3D computer model of 

fnno?l ® shovv® the processing operations performed by surface modeller 80 at step S4-1 6 

[0091] Referring to Figure 8, at step S8-2. silhouette approximator 82 processes the segmented imaae data 
previously generated by image data segmenter 60 at step S4-8 and image segmentaSn ediS 5^ at s^^^ 
approximate the silhouette (that is, the outline) of the subject object in each input mage Sh st^LqhTliSs and 
to assign a unique label to each straight line. In this embodiment, to approximate thrsubj^t oWect Shr^^^ 
each image with straight lines, silhouette approximator 82 performs process^g in rconventSafmannS S extmn e 

St^p^Smn.f ^1°.="^"^ T^^'^J P^^-^^s^'^g 0* i^P-Jt "Tiage 302 in Figure 3. It should belted how^™ Ta" 
Lr» ch^t f ^ °* ^'^T ® ^^"'^.'^ ^" subsequent Figures which show silhouettes), the number of straight ^63 ,^^2 
Z,S<1ZoZTsS11bI" '""^ '"^^ ''^ ^""^"y be generatedln SlbyThotene 

[0093] Referring to Figure 10, at this stage in the processing, the silhouette 500-514 in each input imaae has 

2S been approximated by straight line segments, and the position and orientation of each sHhSe Tn thr^^f 
dirnensional space s known as a result of the calculations perfomied at step S4-6 by camera ral^^^^^^^ 
determine the posrtion and orientation of each input image (each silhouette 500 514 jL ^the ?S dimensi?nS 
plane of an input image . In addition, the position of the focal point of the camera 2^r) in thr«» Hlm^cr^T=T 
IS known tor each input image as a result of L calculations pe^oS S im^olZuSS^rVat'^^^^^^ ''''' 

^ n ^Ih »h ,K s''^°"^"® *°9ether with the focal point 51 5-51 9 of the camera 230 which recorded the imaae 

'".^'f l^^JtT^"^^, ^" '=°"e in three-dimensfonal space which touches t^e See oT^he 

Sf^i-, ^Sf-^t'T'^H"' IW <>' silhooenes 500-514 dairies Ihs bounding airlaoe of 

« SjJ^SfL.*' ? Jres^lraenaonal space. Consequently, a three-dimensional computer model of the sSaS 
ffitS •'^?'*'«"^,'»''"l«"9'''«"»s»»nofallo(theccmes,thalisthe»isualhul ' 
[MWl In this ernbodimera, the Inlersectton of the rjonos is calculated by calculatino the Intersection, of th. 

f£ iCtMrfanglersyi.^^^^^^^^^^^ *Tn -Pe«iX silhoSe^ meeT is "Sat°Jd' in 

Slsl30Tn'd 5?rmeS.^ ""^ '^'"9 - 

?i?f ^..h -^f ar®'^*®"^ computer model accurately represents the surface of the subiect obiect 

210, each intersection of triangles which results In a vertex of the surfar*. nf th-T ct.Kit^f . 

total number of silhouettes is ••m" and the total number of straight lines in the silhouette^i^^"en 0(1^ 



computations would be required. In addition, much of the computation will be unnecessary because many of the 
triangles tested will not intersect (and hence will not lead to the generation of a 3D point) and because 3D points 
which do not lie on the surface of subject object 210 may be generated by chance intersection of triangles (so that 
the 3D points will need to be processed further to determine which ones actually represent vertices for the model of 
the subject object 21 0). 

[0098] Consequently, in this embodiment, processing is performed in a way which reduces the number of 
computations necessary to calculate 3D points representing vertices of the subject object surface. 
^ [0099] Before describing this processing in detail, the principles of the processing will be described. 

[0100] More particularly, in this embodiment, processing is performed to consider a volume of 3D space and to 
test the volume to determine whether: 

(1) the volume can be discarded, either because there are not enough planar triangles in the volume (that Is, 
10 three in this embodiment) to Intersect to generate a 3D point, or because the planar triangles in the volume 

cannot intersect at a 3D point which represents a vertex of the surface of the subject object 210 (as will be 
explained in more detail belov\^; 

(2) the volume is to be subdivided to consider smaller volumes, because the volume contains enough planar 
triangles to intersect and generate more than a predetermined number of 3D points (In this embodiment one 3D 

'5 point) which may represent a vertex of the surface of the subject object 210; or 

(3) the volume contains the correct number of planar triangles (three In this embodiment) to intersect and 
generate a sufficiently small predetermined number (one in this embodiment) of 3D points which may represent 
vertices of the surface of the subject object 210, in which case the 3D positions of these points are calculated 
and tested to determine whether they actually represent vertices of the subject object 21 0. 



[0101] In this way, the processing ensures that all vertices of the surface of the subject object 210 defined by 
the Intersection of the silhouette cones are calculated, while avoiding unnecessary processing which does not lead 
to the calculation of such a 3D point. In particular, large volumes of 3D space can be discarded without further 

25 calculation. 

[0102] In this embodiment, the tests on a volume of 3D space to determine whether it is to be discarded, 
subdivided or a 3D point calculated are carried out by projecting the 3D volume into each two-dimensional image, and 
testing the 2D projection of the 3D volume against the straight lines making up each silhouette in the images 
(previously generated at step S8-2). By performing the tests In 2D in this way, the number of processing operations 
which need to be performed to test the volume Is reduced. 

^0 [0103] To determine whether a volume can be discarded because the planar triangles therein cannot Intersect at a 
3D point which represents a vertex of the surface of the subject object 210, processing is perfonned to determine 
whether the 3D point will lie wrthin all of the cones (polyhedra) defined by the silhouettes not contributing a 
plane which intersects at the 3D point. Thus, referring to the example shovwi In Figure 10, in the case of planes 530, 
532 and 534 processing Is performed to determine whether the 3D point 536 will lie within all of the cones defined 

55 by silhouettes 500, 502, 504, 506, 508 and 510 (because these silhouettes do not define one of the three planes 
intersecting at the 3D point 536). In this embodiment, this processing is carried out by testing the 2D projection 
of the volume in each image to ensure that the 2D projection lies inside each silhouette. That is, the tests are 
performed in 2D, which reduces the processing operations which need to be performed. 

[0104] In this embodiment, when a 3D point is calculated, It Is subsequently tested to determine whether it 
represents a vertex of the subject object 210. This Is because a 3D point is calculated when a 3D volume is 
40 Identified which contains three planar triangles. However, the three planar triangles may not actually intersect at a 
3D point within the volume. Accordingly, a test Is carried out in this embodiment to determine vsliether the 
calculated 3D point is within the identified volume. 

[0105] The processing in this embodiment assumes that the surface of the subject object is a simple polyhedron, 
that is, there are no coincidences v\4iere more than three planes meet at a single 3D point. In computer graphics 
applications, this is a trivial restriction because this situation can always be obtained, without affecting the 
visual result, by adding a small amount of random noise to the data defining the straight lines approximating each 
silhouette, if necessary. 

[0106] The processing performed in this embodiment to calculate 3D points representing vertices of the surface 
of subject object 210 will now be described in detail. 

[0107] Referring again to Figure 8, at step S8-4, silhouette approximator 82 processes the data generated at 
50 Step S8-2 for each silhouette to calculate a bounding box for each straight line segmerit therein and to store the 
bounding box data in a respective quad-tree structure for each silhouette. 

[0108] This processing Is performed prior to the calculation of any 3D vertex of the surface of the subject 
object 210 because, as will be explained later, the data resulting from step S8-4 is used to determine which 
straight lines in a silhouette do not need to be considered during processing to calculate 3D points because the 
processing of such straight lines cannot generate 3D vertices of the surface of the subject object 210. 
[0109] Figure 1 1 shows the processing operations performed by silhouette approximator 82 at step S8-4. 



^ *° '''9^'® ^* ^'®P Si 1-2, Silhouette approximator 82 considers the next silhouette to be 

proc^sed (this being the first silhouette the first time step 811^2 is performed) and at stL Si 1 f r^fr^l 1 ! 

S^rgtL'S^^TLt'lli'" '° '"'^^ "'^^ ^^'^'^ strS'?reVp?oximaStSrsio'S«^^^^^^^^ 

[0111] More particularV, referring to Figure 12 by way of example, silhouette approximator 82 calculates a 

the bounding box is defined to have zero width in the direction perpendicular tfthj line (Shouah Se bound no 

[01 12] Silhouette approximator 82 stores the minimum and maximum x values and the minimum and maximum v 
CllVSi^ZSt'^^^^^^^ °* ' conventional puad-tree (these values being inXtrfo7=ng^ 

SJrS in th«^L''lH^t^^^'®'^^"*'°''^"1 approximator 82 detemnines the range of x values and the range of y values 
tfr^^ lln Qi.T node currently being considered (this being the root node generated at step S11-4 the fTs^ 
hP «=mlTJl:=i'^ ^*®P S^^-^' approximator 82 ^Iculates the difference between 

he smallest x va ue and the largest x value, and the difference between the smallest y value and the largest y vilue 
[0114] At step S1 1-8 silhouette approximator 82 creates two child nodes in the quad-tree by splitting the node 

f.r^^rt tSnL"^T^°?^-"^%^. ""^^'^^ °* ^^'"^^ determined at step L i -6 to have °he 

largest range. That is, if it is detemiined at step Si 1-6 that the range of y values is larger than the ranae of x 

miS^ iffheTvaS"'"''""'"' °* ^ °the.lisS S fe madi at *e 

[0115] Tlius, referring to the example shown in Figure 12, because the range of v values is oreater than th*. 
range o • x values, silhouette approximator 82 creates two child nodes 560 and 562 by defining T llS sM 
perpendicular to the y axis v^*iich passes through the median value of the y values (thafis the medSn line seSmen^ 
Kne 56Tl°H5n/H "^P'® °' there are seventeen line end points, and acJCrding^th^^^ 

S anH tinh . ° ^""^ ^'^'^^ *®^« ^'Sht line Segment end points above the I ne 

Pmhnr^^L!.?^^!^ tf^"'®"* ^"'^ ^®'°^ 'f^® 564. If there is an even number of values S in hij 

ftTa^cSrihe mSS 564 to pass through the mid-point of the two vi ^iS 

£!InJnnH«^l*hI^''h u ^■!?' ^PP'°''''"^^°' allocates bounding boxes calculated at step Si 1-4 from the 

parent node to be child nodes created at step S1 1-8. y • •* num me 

S^oSiallv v!^Wn Sfi^'Slfn' noH'^°"^f /f''T'"'^l°' ^ '^""^'"9 *° ^ c^^''^ node if it lies wholly 

?2 parh hn^H no K ^ ^"^^i "'■J* '* ^l^'^^ "^'^ "^^"5. referring to the example shown in Figure 

hPinw r^.^°\'^l?? «^°"y above line 564 is allocated to child node 560, and each bounding box lying 3y 
J,H^H-?f ^ Lo' u''^*^ *° ''^''f ^^2. Bounding boxes 570, 572 and 574 are allocated to both child nodrseo 
and child node 562 because each of these bounding boxes lies partially in child node 560 and partially in cES node 
hf • 5°""*^"^ ^""^t ^^i^"^ 578 are allocated to both child node 560 and child node 562 becSuJe ekch of thS 
bounding boxes touches the line 564 defining the boundaiv of the child nodes 560 and 562 

S,IS^ *u ^t" V:^^' !'."i°"e"e approximator 82 considers the next child node, and at step Si 1-14 determines 
wj^^her the number of bounding boxes in the child node is the same as the number of bounding boxes in the ^ent 

Eri?l th» H'! ^'^"7"^^ ^i^PS^.^-''^ ""'"ber of bounding boxes in the child node and the parent 

^vi k1 same *hen further sub-division of the child node is unnecessary because separation of the bounding 
boxes between further child nodes v^uld not be possible (each further child note would contain all of the bound no 
&..IT.'^? ^T^- ^^l?'''"?.'- """e'^essary to keep the current child node considered at step SlT?^ 

creTt^ a s^Tsi 1 8%^\Ti. ^'^^1 *f ^"'^ ""necessary to keep the otheVchiW noJe 

created at step 51 1-8 (that is, the sibling of the child node considered at step 811-12) because that child nnri*. 

con1id;rldTst;;iri-?2^^^^ °* ^^'^^^ '^^-^ P^'-"' are^in thllSrrenl Slid nSde 

l?J = Accordingly, when it is determined at step Si 1-14 that the number of bounding boxes in the child node is 
the same as the number in the parent node, processing proceeds to step S11-16. at which silhouette aoDroxi^ator 82 

pSSLCsii'Io.""'''"' '''' "■''""3 "''^ ^ Se?lr'TSsZgVen 

if ^-l'' ^ °" ^^"''■J* " '® determined at step Sl 1-14 that the number of bounding boxes in the child node 

IS not the same as. the number in the parent node, processing proceeds to step Sll 18, at which si hoJette 
approximator 82 detemr,.nes vv*,ether the number of bounding boxes in the child node is equal to two silhouette 
th^n^nrth.. It IS determined at step 811-18 that the number of bounding boxes in the child node is equal to two 
^1 ^ ° *® "°«^e 's not performed in this embodiment because a child node has a°reS 

been obtained wrth a sufficiently small number of bounding boxes in it. Accordingly, processing proceeds to step In^ 

[0123] On the other hand, if it is determined at step Si 1 -18 that the number of bounding boxes in the child node 



is not equal to two, then processing returns to step S11-6. Steps S11-6 to S11-18 are repeated to repeatedly create 
and subdivide child nodes until it is determined at step S11-14 that the number of bounding boxes in a child node 
and its parent node are the same or until It Is determined at step Si 1-18 that a child node has been generated 
containing two bounding boxes. 

[0124] At step S1 1 -20, silhouette approximator 82 determines whether there is another child node to be processed, 
and steps Si 1 -12 to Si 1 -20 are repeated until each child node has been processed In the way described above. 
[0125] At step S1 1 -22, silhouette approximator 82 determines v\^ether there is another silhouette to be processed, 
and steps S1 1-2 to S1 1-22 are repeated until each silhouette has been processed In the way described above. 
[0126] Referring again to Figure 8, at step S8-6, 3D point calculator 84 perfomis processing to calculate 3D 
points representing vertices of the surface of the subject object 210. 

[0127] Figure 1 3 shows the processing operations perfonned by 3D point calculator 84 at step S8-6. 
[0128] Referring to Figure 13, at step S13-2, 3D point calculator 84 defines a volume in 3D space in which the 
subject object 210 lies, and hence in which ail calculated 3D points representing points on the object surface will lie. 
[0129] Referring to Figure 14a, in this embodiment, the start volume defined by 3D point calculator 84 at step S13- 
2 comprises a cuboid 600 having vertical side faces and horizontal top and bottom faces. The vertical side faces are 
positioned so that they touch the edge of the calibration pattern of features on the photographic mat 34 (and 
therefore wholly contain the subject object 210). The position of the top face of the cuboid 600 is set at a 
position defined by Intersecting a line 610 from the focal point 612 of the camera 230 through the top edge of any 
one of the input images stored at step S4-4 with a vertical line 614 through the centre of the photographic mat 34. 
More particularly, the focal point 612 of the camera 230 and the top edge of an image are known as a result of the 
position and orientation calculations performed at step S4-16 and, by setting the height of the top face to 
correspond to the point where the line 610 intersects the vertical line 614 through the centre of the photographic 
mat 34, the top face will always be above the top of the subject object 210 (provided that the top of the subject 
object 210 is visible In each input image). The position of the horizontal base face of the cuboid 600 Is set at a 
position defined by intersecting a line 616 from the focal point 612 of the camera 230 through the bottom edge of 
any one of the input images stored at step S4-4 with the vertical line 614 through the centre of the photographic mat 
34. This ensures that the base of the cuboid 600 extends below the bottom of the subject object 210 (provided that 
the base of the subject object 210 is visible in each input image). 

[0130] In this embodiment, by imaging the subject object 210 as illustrated in Figure 2 in which camera 230 is 
mounted on a tripod in a fixed position and the photographic mat 34 with subject object 210 thereon Is moved 
relative to the camera, the camera 230 will always be outside the cuboid 600 when each image was recorded. However, 
instead of moving the photographic mat 34 and subject object 210 relative to stationary camera 230, it is possible 
to move camera 230 to different positions and orientations while leaving the photographic mat 34 and subject object 
210 stationary. In this case, as illustrated in Figure 14b, the position of camera 230 may be within cuboid 600 when 
an image is recorded to show the top of the subject object 210 from an elevated vievN^Doint. Since the processing to 
be performed by 3D point calculator 84 in this embodiment projects the cuboid 600 Into the images recorded by camera 
230 to detemnine information about what is contained in the projected volume, the cuboid 600 needs to lie 
completely in front of the camera 230 for all recorded images. Consequently, In this embodiment, 3D point 
calculator 84 tests the cuboid 600 against the camera positions and orientations calculated at step S4-6 and, as 
illustrated in Figure 14c, adjusts the cuboid 600 so that each corner thereof is in front of every calculated camera 
position. 

[0131] Referring again to Figure 13, at step S13-4, 3D point calculator 84 adds the 3D volume defined at step 
S13-2 (that is, cuboid 600 in the example of Figures 14a, b and c) to the top of a storage stack, and at step Si 3-6 
pops the next volume off the top of the stack (this being the initial volume defined at step Si 3-2 the first time 
step S13-6 is performed). 

[0132] At step Si 3-8, 3D point calculator 84 determines the state of the volume 

popped off the stack at step S1 3-6. 

[0133] In this processing, 3D point calculator 84 determines whether the volume popped off the stack at step S13- 
6 cannot contain any 3D vertex of the surface of the subject object 210 (and hence can be discarded), VN^ether the 
volume needs to be subdivided because rt may contain more than one 3D vertex of the surface of the subject object 
210, or whether the volume contains planes which may define just a single vertex of the subject object surface, In 
which case the position of the 3D vertex can be calculated and tested. In this embodiment, 3D point calculator 84 
performs these determinations by projecting the 3D volume popped off the stack at step Si 3-6 into each Image and 
testing the resulting 2D projection against the straight lines approximating each silhouette previously calculated 
at step S8-2. 

[0134] Figure 15 shows the processing operations performed by 3D point calculator 84 at step Si 3-8. 
[0135] Referring to Figure 15, at step Si 5-2, 3D point calculator 84 projects the corners of the volume popped 
of the stack at step S13-6 into the two-dimensional imaging plane of camera 230 for each imaging position and 
orientation previously calculated at step S4-6, and retains data defining the connectivity of the projected comers. 
[0136] The result of this projection, which is illustrated in Figure 16 for one image, is a 2D projection 650 
comprising eight points 620-634 defining six enclosed regions within the two-dimensional imaging plane (it being 
noted that these regions may overlap). In the example of Figure 16, the six regions defined by the points are the 
regions defined by the groups of points (620, 622, 624. 626). (620, 622, 630, 628). (622. 630, 632, 624). (624, 632, 
634, 626), (626, 620. 628. 634), and (628, 630. 632. 634). 
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list and the prcx:essing to be performed on the volume popped off the stack is detemiined based on the higher 

priorrty case. 

[0153] VOLUME STATE CASE A : Set state to "discard the volume" if any image has relationship 8 (described 
above with reference to Figure I7h) between the silhouette and the 2D projected volume 650. In relationship 8, the 
2D projected volume 650 and the silhouette 724 do not overlap, and accordingly any 3D points in the volume popped 
off the stack at step S13-6 do not project to a point on or within the silhouette 724 and consequently cannot 
represent a vertex of the surface of subject object 210. Accordingly, the volume popped off the stack at step S13-6 
is to be discarded. 

[0154] VOLUME STATE CASE B : Set state to "calculate a 3D point" if 3. and only 3, images have 
relationship 4 (described above with reference to Figure 17d) between the silhouette and the 2D projected volume 650, 
and the relationship in all of the other images Is relationship 7 (described above with reference to Figure I7g). 
[0155] As described previously with reference to Figure 10, a 3D vertex of the surface of subject object 210 is 
defined by the intersection of three planar triangles, each from a different silhouette, or by the intersection of 
two triangles from the same silhouette with one triangle from a further silhouette. 

[0156] The case where three triangles from different silhouettes intersect at a 3D point is identified in this 
embodiment when three silhouettes having relationship 4 with the projected volume 650 described above with reference 
to Figure 17d are found and the relationship between the silhouette in all other images and the projected volume 650 
is relationship 7 described above with reference to Figure 17g. More particularly, relationship 4 identifies that 
there is only a single edge of a given silhouette in the 2D projected volume 650 and accordingly three silhouettes 
having this relationship with the projected volume 650 identify three planar triangles (defined by the three 
respective edges which intersect the 2D projected volume 650) which may intersect at a 3D point in the volume popped 
off the stack at step S13-6. In addition, because the relationship between the silhouette and the projected volume 
650 In every other image is relationship 7, the 3D point defined by the intersection of the three planar triangles 
projects to a point inside the silhouette in all of the images and accordingly represents a vertex of the surface of 
the-subject object 210 (provided that the 3D point actually lies In the volume popped off the stack at step S13-6 
and does not lie outside this volume). 

[0157] VOLUME STATE CASE C : Set. state to "calculate a 3D point" if 1 (and only 1) image has relationship 
1 (described above with reference to Figure 17a) between the silhouette and the 2D projected volume 650, 1 (and only 
1) image has relationship 4 (described above with reference to Figure 17d) and the relationship in all of the other 
images is relationship 7 (described above with reference to Figure 17g). 

[0158] lh\s case C represents the case where two planar triangles from the same silhouette and a planar triangle 
from a different silhouette meet at a 3D point. More particularly, the relationship described above with reference 
to Figure 17a, in which only a single silhouette vertex is present in the 2D projected volume 650, defines the two 
planar triangles from the same silhouette which meet at the 3D point (these planar triangles being defined by the 
silhouette edges which meet at the vertex 700 in the projected volume 650). The relationship described above with 
reference to Figure I7d defines the planar triangle from the second silhouette which meets at the 3D point (that is, 
the planar triangle which passes through the end points of the straight line which intersects the 2D projected 
volume 650). As noted above, because the relationship between the silhouette and the projected volume 650 In all 
other images is relationship 7 described above with reference to Figure I7g, the 3D point defined by the 
Intersection of the two planar triangles from the first silhouette and the planar triangle from the second silhouette will 
project to a point within the silhouette in all other images, and accordingly will represent a vertex of the s^urface 
of subject object 210 (provided again that the 3D point actually lies within the volume popped off the stack at step 
S13-6). 

[0159]. VOLUME STATE CASE D : Set state to "discard the volume" if any of the following occur, all of 
which indicate that the volume popped off the stack at step S13-6 does not contain a sufficient number of planar 
triangles (that is, three) to intersect and define a 3D point which may lie on the surface of the subject object 210 
(and consequently the volume is to be discarded): 

',(1) 1 (and only 1) image has relationship 1 (described above with reference to Figure 17a) between the 
silhouette and 2D projected volume 650, and the relationship in all of the other Images Is relationship 7 
(described above with reference to Figure 17g); 

(ii) 1 or 2 (but no more than 2) images have relationship 4 (described above with reference to Figure 17d) 
between the silhouette and 2D projected volume 650, and the relationship in all of the other images is 
relationship 7 (described above with reference to Figure 17g); 

(iii) all of the images have relationship 7 (described above with reference to Figure 17g) between the 
silhouette and the 2D projected volume 650. 



[0160] VOLUME STATE CASE E : set state to "discard the volume" if 1 (and only 1) image has relationship 3 
(described above with reference to Figure 17c) or relationship 5 (described above with reference to Figure 17e) 
between the silhouette and the 2D projected volume 650, and the relationship in all of the other images is 
relationship 7 (described above with reference to Figure 17g). In this case, only one image contains a silhouette 
vertex or edge within the 2D projected volume 650. However, In order for volume state case B or volume state case 0 
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currently being considered overlaps the bounding box 800 of the projected volume generated at step S18-2. That is, 
3D point calculator 84 tests whether any part of the bounding box 800 of the projected volume lies within the region 
defined by the node. 

[0174] Referring to Figure 19 by way of example, three regions 810, 812, 814 of the image are shown. The root 
node of the quad-tree comprises the whole of the image (that Is, regions 810, 812 and 814) and therefore there is an 
overlap between the root node and the bounding box 800. 

[0175] A first child node of the root node comprises region 810 and a second child node of the root node 
comprises region 812 plus region 814. In both cases, there is an overlap between the child node and the bounding box 
800. However, when the child nodes of the second child node are considered (that is, region 812 and region 814), 
there Is an overlap between the bounding box 800 of the projected volume and node 814, but there Is no overlap 
between the bounding box 800 and node 812. 

[0176] If it is determined at step S18-6 that the node currently being considered does not intersect the 
bounding box of the projected volume, then at step S18>8, 3D point calculator 84 removes any nodes which are 
children or further descendants of the node currently being considered from the list of nodes remaining to be 
processed (because the children and further descendants all lie within the parent and hence cannot intersect the 
bounding box of the projected volume if the parent does not intersect it). Thus, referring to the example in Figure 
19, when node 812 is considered at step Si 8-4 and it is found at step S18-6 that the node does not overlap the 
bounding box 800 of the projected volume, then at step S18-8, any children and further descendants of the node 812 
are removed from the nodes remaining to be processed, thereby reducing the amount of processing to be carried out. 
[0177] On the other hand, if it is determined at step S18-6 that the node currently being considered does 
overlap the bounding box 800 of the projected volume, then, at step S18-10, 3D point calculator 84 determines 
whether the node has any children. 

[0178] If it is detemnined at step Si 8-10 that the node does have children, then at step S18-12, the next child 
node is considered and processing returns to step S18-6 to determine whether the child node overlaps the bounding 
box of the projected volume. Steps Si 8-6, S18-10 and S18-12 are recursively performed until a node in the quad-tree 
is reached which overlaps the bounding box 800 of the projected volume and which does not have any child nodes. 
When such a node is reached, processing proceeds to step S18-14, at which 3D point calculator 84 tests the bounding 
box of each edge in the node to determine whether it intersects the bounding box 800 of the 2D projected volume 650. 
3D point calculator 84 then labels as a "candidate edge" each edge In the node having a bounding box which does 
intersect the bounding box 800 of the projected volume 650. 

[0179] At step Si 8-1 6, 3D point calculator 84 determines whether there are any more nodes in the quad-tree, and 
steps S18-4 to S18-16 are repeated until each node has been processed in the manner described above. 
[0180] In this v\ay, 3D point calculator 84 discards every node in the quad-tree which does not intersect the 
bounding box of the projected volume and labels the silhouette straight line segments in the remaining nodes as 
."candidate edges" (that is, edges which may intersect the 2D projected volume 650 Itself). 

[0181] Referring again to Figure 15, at step S15-18, 3D point calculator 84 determines which candidate edge end 
points (that is, the point at which a first candidate edge identified at step Si 5-1 6 meets a second candidate edge 
in the silhouette) lie within the 2D projected volume 650, and labels such end points as "verified vertices". The 
processing at step S15-18, therefore, determines whether there are any points such as the points 700, 702, 706, 708, 
710 and 712 in Figures 17a, 17b and 17c within the projected volume 650 (these points being labelled as "verified 
vertices"). 

[0182] Figure 20 shows the processing operations performed by 3D point calculator 84 at step Si 5-1 8. 

[0183] Referring to Figure 20, at step S20-2, 3D point calculator 84 sets a counter "nw" representing the 

number of verified vertices to be equal to 0. 

[0184] At step S20-4, 3D point calculator 84 considers the second end point of the next candidate edge (this 
being the first candidate edge the first time step 820-4 is performed). In subsequent processing, the end point is 
tested to determine whether it lies within the 2D projected volume 650. It is only necessary to consider and test 
each second end point because the end points are ordered such that each silhouette (which is closed) consists of the 

edges (pi, p2), (p2, p3), , (pn, pi), where pi to pn are the end points. Thus, if a line segment between the 

end points (p, q) is a candidate edge, then there will be an edge between the end points (x, p) v\^ich is also a 
candidate edge, for some x, because the bounding boxes of these straight line segments in the silhouette intersect 
at the point p, and accordingly it is not necessary to test the first end points. 

[0185] At step S20-6, 3D point calculator 84 determines whether the end point currently being considered is 
within the 2D projected volume 650. In this embodiment, this is carried out by determining whether the end point is 
within any of the six projected faces of the volume by testing the end point against each projected face using a 
conventional method, such as one of the methods described in "Graphics Gems" ^ IV 
pages 16-46, ISBN 0-12-336155-9. 

[0186] If it is determined at step S20-6 that the end point is within the 2D projected volume, then at step S20- 
8, 3D point calculator 84 labels the end point as a "verified vertex", and at step S20-10 increments the value of 
the counter nvv by 1 . 

[0187] On the other hand, if it is determined at step S20-6 that the end point is not within the 2D projected 
volume, then steps S20-8 and S20-10 are omitted. 

[0188] At step S20-1 2, 3D point calculator 84 determines whether the value of the counter nw is greater than 1 . 
[0189] If it is determined at step S20-12 that the value of the counter nw is greater than 1, then the 
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[0209] At step S21 -6, 3D point calculator 84 determines whether any part of the candidate edge selected at step 
S21-4 intersects the 2D projected volume 650. 

[0210] More particularly, in this embodiment, at step S21-6, 3D point calculator 84 tests the candidate edge 
against each of the twelve edges of the 2D projected volume 650 using a conventional intersection test method, such 
as the one described In "Graphics Gems" II pages 7-9, ISBN 0-12-064481-9. 

[0211] If It Is determined at step S21-6 that the candidate edge intersects the 2D projected volume 650, then at 
step S21-8, 3D point calculator 84 labels the candidate edge as a "verified edge", and at step S21-10 increments the 
value of the counter nve by 1 . 

[0212] On the other hand, if it Is determined at step S21-6 that the candidate edge does not intersect the 2D 
projected volume 650, then steps S21-8 and S21-10 are omitted. 

[0213] At step S21-12, 3D point calculator 84 determines whether the value of the counter nve is greater than 1 . 
[0214] - If it is determined at step S21-12 that the value of counter nve is greater than 1, then the processing 
proceeds to step S21-14, at which a flag is set to indicate that more than one verified edge has been identified. 
[0215] On the other hand, if it is determined at step S21-12 that the value of the counter nve is not greater 
than 1, processing proceeds to step S21-16, at which 3D point calculator 84 detemnines whether there is another 
candidate edge to be processed. 

[0216] Steps S21-4 to S21-16 are repieated until it is determined at step S21-12 that the value of the counter 
nve is greater than 1 or until it is determined at step S21 -1 6 that all candidate edges have been processed. 
[0217] When it Is determined at step 821-16 that all of the candidate edges have been processed, processing 
proceeds to step 821-18, at which 3D point calculator 84 reads the value of the counter nve and sets a flag to 
indicate that the number of identified verified edges is 0 or 1, depending upon the number of verified edges defined 
by counter nve. 

[0218] Referring again to Figure 15, step 815-40, 3D point calculator 84 reads the value of the flag set at step 
821-14 orstep S21-18 to determine the number of verified edges. 

[0219] If there is more than one verified edge, the relationship between the silhouette and the 2D projected 
volume 650 is relationship 5 described above with reference to Figure 17e (it having previously been estatilished at 
step 815-20 that there are no silhouette edge vertices in the 2D projected volume so that the relationship cannot be 
relationship 2 described above with reference to Figure 17b, relationship 3 described above with reference to Figure 
17c or relationship 6 described above with reference to Figure 17f). In this case, processing proceeds to step 815- 
42, at which 3D point calculator 84 increments the value of counter ne defining the number of active cameras in 
which at least one verified edge has been detected by 1 . 

[0220] At step 815-44, 3D point calculator 84 sets the flag eitherSubdivideOrDiscard to "true". This is because 
volume state case B or volume state case C described above (which result in a 3D point being calculated) cannot 
arise if the relationship between any silhouette and the 2D projected volume 650 is relationship 5 described above 
with reference to Figure I7e. Accordingly, the volume popped off the stack at step S13-6 is either to be subdivided 
or discarded depending on the relationships between the 2D projected volume 650 and the silhouettes in the other 
Images. 

[0221] At step 815-46, 3D point calculator 84 increments the value of counter np defining the number of planes 
intersecting the volume popped off the stack at step 813-6 by 4 so as to increase the value of the counter beyond 
the threshold value of 3 which will be used in subsequent processing. 

[0222] Returning again to step 815-40, If it is determined that the number of verified edges is 1, the 
relationship between the silhouette currently being processed and the 2D projected volume 650 is relatiori'ship 4 
described above with reference to Figure 17d. In this case, processing proceeds to step 815-48, at which 3D point 
calculator 84 stores data defining the current silhouette. 

[0223] At step 815-50, 3D point calculator 84 increments the value of counter ne defining the number of active 
cameras in which at least one verified edge has been detected by 1, and at step 815-52 increments the value of 
counter np defining the number of planes intersecting the volume popped off the stack at step 813-6 by 1 . 
[0224] At step 815-54, 3D point calculator 84 determines whether the value of 

counter np is less than or equal to 3. 

[0225] If It is determined at step 815-54 that the value of counter np is less than or equal to 3, then rt Is 
still possible that volume state case B or volume state case C described above (which result in the calculation of a 
3D point) can arise and accordingly at step Si 5-56, 3D point calculator 84 stores data defining the verified edge. 
[0226] On the other hand, if it is determined at step 815-54 that the value of counter np is greater than 3, 
neither volume state case B nor volume state case C described above can arise, and accordingly step 815-56 Is 
omitted. 

[0227] After step 815-56, or when it is determined at step 815-54 that the value of counter np is greater than 3, 
processing returns to step 815-12 to consider the next silhouette, if any. 

[0228] Returning again to step 815-40, if the number of verified edges Is 0, the relationship between the 
silhouette currently being processed and the 2D projected volume 650 Is either relationship 7 described above with 
reference to Figure 17g or relationship 8 described above with reference to Figure 17h. In this case, processing 
proceeds to step 815-58, at which 3D point calculator 84 selects an arbitrary point on the 2D projected volume 650. 
More particularly, In this embodiment. 3D point calculator 84 selects a point corresponding to one of the projected 
volume corners. 

[0229] At step 815-60, 3D point calculator 84 determines whether the point on the 2D projected volume 650 
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[0242] On the other hand, if it is determined at step S 15-74 that the sum nv + ne is greater than or equal to 2, 
processing proceeds to step S15-78, at which 3D point calculator 84 determines whether the sum nv + ne is greater 
than 2. 

[0243] If it is determined at step S15-78 that the value of the sum nv + ne is greater than 2, then volume state 
case F described above has been identified, and processing proceeds to step S15-80, at which 3D point calculator 84 
sets a flag to indicate that the volume popped off the stack at step S13-6 is to be subdivided and the subdivided 
volumes processed further. 

[0244] On the other hand, if It is determined at step S15-78 that the sum nv + ne Is equal to 2, processing 
proceeds to step S15-82, at which 3D point calculator 84 determines whether the value of counter nv defining the 
number of active cameras In which at least one verified vertex has been detected is equal to 0. 

[0245] If it Is determined at step S15-82 that the value of counter nv Is equal to 0, then volume state case G 
has been identified, and processing proceeds to step S15-84, at which 3D point calculator 84 sets a flag to indicate 
that the volume popped off the stack at step S13-6 is to be discarded. 

[0246] On the other hand, if it is determined at step S15-82 that the value of counter nv is not equal to 0, 
then volume state case H has been identified, and processing proceeds to step S15-86, at which 3D point calculator 
84 sets a flag to indicate that the volume popped off the stack at step S13-6 Is to be subdivided and the resulting 
smaller volumes processed further. 

[0247] As a result of the processing described above with reference to Figure 15, 3D point calculator 84 has 
determined the volume state case which exists for the volume popped off the stack at step Si 3-6 and has set a flag 
in dependence upon the determined volume state case Indicating what subsequent processing Is to be performed on 
the volume. ** 

[0248] Referring again to Figure 13, after performing processing as described above to determine the state of 

the volume, at step S1 3-10, 3D point calculator 84 detemnines whether the state is "calculate vertex". 

[0249] If it is determined at step S13-10 that a vertex Is to be calculated, processing proceeds to step S13-12 

to calculate the vertex, otherwise step Si 3-1 2 is omitted. 

[0250] Figure 22 shows the processing operations performed by 3D point calculator 84 at step Si 3-1 2 to calculate 
a vertex. 

[0251] Referring to Figure 22, at step S22-2, 3D point calculator 84 considers the three silhouette edges which 
intersect the current volume (that is, the edges identified in the data stored at steps S15-36 and/or S15-56), and 
calculates the 3D point of intersection of the three planes each of vs4iich contains a respective one of the edges and 
the focal point of the camera 230 when the image containing the edge was recorded. In other words, 3D point 
calculator 84 calculates the 3D point of intersection of the planes which contain the planar faces defined by 
projecting rays from the camera focal points through the end points of the stored silhouette edges. 
[0252] Thus, referring to the example in Figure 10, the 3D point of Intersection 528 of the three planes 
containing the planar faces 522, 524 and 526 Is calculated. This processing Is performed using a conventional plane 
intersection algorithm. 

[0253] At step S22-4, 3D point calculator 84 tests whether the 3D point calculated at step S22-2 Is inside the 
volume popped off the stack at step S13-6. This test is necessary because the planes Intersecting the volume need 
not necessarily intersect each other at a 3D point in the volume. 

[0254] If it is determined at step S22-4 that the calculated 3D point is inside the volume, then processing 
proceeds to step S22-6, at which 3D point calculator 84 adds the calculated 3D point to the list of 3D points 
representing vertices of the surface of the subject object 210. 

[0255] On the other hand, if it is determined at step S22-4 that the calculated 3D point is outside the volume, 
then step S22-6 is omitted. 

[0256] At step S22-8, 3D point calculator 84 sets a flag to indicate that the volume popped off the stack at 
step S13-6 is now to be discarded because processing of it is complete. 

[0257] Referring again to Figure 13, at step S13-14, 3D point calculator 84 determines whether the state of the 
volume popped off the stack at step SI 3-6 Is set to "discard" or "subdivide". 

[0258] If the state is set to "discard" (which can result from processing at step Si 3-8 or as a result of step 
S22-8 at step Si 3-1 2), then, at step S13-16, 3D point calculator 84 discards the volume. 

[0259] On the other hand, if the state of the volume is set to "subdivide", at step S13-18, 3D point calculator 
84 subdivides the volume and adds the child volumes to the top of the stack created at step Si 3-4. In this 
embodiment, the subdivision of the volume at step Si 3-18 comprises the binary subdivision of the volume to generate 

eight new child volumes. 

[0260] Following step S13-16 or step S13-18, processing proceeds to step S13-20, at which 3D point calculator 84 
determines whether there is another volume on the stack (including any child volumes added at step S13-18). 
[0261] Steps Si 3-6 to Si 3-20 are repeated until each volume on the stack has been processed in the way 
described above. 

[0262] As a result of performing processing in this way, 3D point calculator 84 has generated a set of points in 
3D space representing vertices of the surface of subject object 210. 

[0263] Referring again to Figure 8, at step S8-8, polygon generator 86 connects the 3D points generated by the 

3D point calculator 84 to generate a polygon mesh representing the surface of subject object 210. 

[0264] Figure 23 shows the processing operations perfonmed by polygon generator 86 at step S8-8. 

[0265] Referring to Figure 23, at step S23-2, polygon generator 86 generates plane data for the polygons In the 
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of the 3D points. More particularly, two 3D points which are to be connected have In their signatures assigned at 
step S25'12 two plane IDs which are common to both signatures. For example, the signature of 3D point V1 in Figure 
27 is {a, b, e} and the signature of 3D point V2 is {a, b, c}. The plane IDS a and b are therefore common to the 
signatures and identify a connection between the 3D points V1 and V2 because they define planes which meet at an 
edge between the 3D points V1 and V2 as shown in Figure 27. 

[0286] As will be explained below, in the processing at step S23-6, polygon generator 86 determines connections 
between the 3D points in each individual polygon by following edges around the polygon determined using the 

^ signatures of the 3D points in the v^ay described above until the connections return to the start 3D point. 

[0287] In the case of a convex subject object, this is the only processing which needs to be performed to 
connect the 3D points because in each set of 3D points generated at step S23-4 (comprising 3D points lying on the 
same plane), there will be only two 3D points having in their signatures tvNo plane IDs which are the same. By way of 
example, referring to Figure'27, the set of 3D points for plane A comprises the points V1, V2, V3 and V4 but only 

70 the points V1 and V2 have the plane IDs a and b in their signatures. 

[0288] However, for parts of a subject object surface which are not convex, there may be more than two 3D points 
which have the same two plane IDs In their signatures, and hence the connections between the 3D points cannot be 
determined based on the signatures alone. 

[0289] By way of example, Figure 28 shows an example In which four 3D points 1000, 1010, 1020, 1030 He on the 
Intersection of two planes 910, 1040. Accordingly, in this example, the signature of each of the four 3D points 1000, 
1010, 1020, 1030 contains the plane IDs of the planes 910 and 1040. 

[0290] To address this problem, In this embodiment, polygon generator 86 performs processing to detemnine the 
connections between the 3D points by connecting the points in pairs based in the order that they occur along the edge 
defined by the intersection of the planes 910 and 1040. This processing will be described in detail below. 
[0291] As will also be explained below, in this embodiment, polygon generator 86 also performs processing to 
2Q address a further problem that can occur with parts of a subject object surface which are not convex, namely the 
problem that more than one polygon representing the object surface may lie in the same plane. 

[0292] A first way in which this can occur is shown in the example of Figure 28, in which the polygons 1050 and 
1060 representing separate parts of the subject object surface both lie in the plane 910. 

[0293] A second way in which more than one polygon may lie in the same plane is illustrated in Figure 29. In 
this example, polygon 1100 represents part of the object surface and polygon 1110 represents a hole in the subject 

25 object surface. 

[0294] The processing performed by polygon generator 86 at step S23-6 will now be described In detail. 
[0295] Figure 30 shows the processing operations performed by polygon generator 86 at step S23-6. 
[0296] Referring to Figure 30, at step S30-2, polygon generator 86 creates an empty polygon set "S" for the 3D 
computer model of the subject object 210, to which polygons representing parts of the surface of the subject object 
210 will be added as they are calculated. 

[0297] At step S30-4, polygon generator 86 considers the next plane "p" defined at step S23-2, and reads the 3D 
points in the set Vp generated at step S23-4 for the plane. 
[0298] At step S30-6, polygon generator 86 determines whether there are any 3D points in the set Vp read at step 
S30-4. On the first iteration of the processing, 3D points will exist in the set Vp, and processing proceeds to step 
S30-8. However, in subsequent processing, 3D points are deleted from the set Vp after connections for the point have 
35 been calculated. Accordingly, on subsequent iterations, It may be determined at step S30-6 that no more 3D points 
remain in the set Vp, and in this case, processing proceeds to step S30-50, which will be described later. 
[0299] At step S30-8, polygon generator 86 adds a new polygon data structure "s" to the polygon set "S" created 
at step S30-2, Into which data defining a polygon will be Input as it is generated. Also at step S30-8, polygon 
generator 86 stores data defining the normal vector previously calculated at step S25-6 for the plane currently 
being considered in the polygon data structure "s" to define a normal for the polygon. 
^0 [0300] At step S30-10, polygon generator 86 selects a 3D point from the set Vp as a start vertex "u" for the 
polygon (any of the 3D points may be selected as the start vertex "u"). 

[0301] At step S30-12, polygon generator 86 sets a current vertex pointer 'W to the vertex "u" to Indicate that 
this is the vertex currently being processed, and at step S30-14, reads the signature of the start vertex "u". 
[0302] At step S30-16, polygon generator 86 selects a plane "q" defined in the signature read at step S30-14 
45 which is not the same as the plane "p" currently being considered (selected at step S30-4). 

[0303] To assist understanding of the processing performed by polygon generator 86, reference will be made to 
the example shown in Figure 27. Referring to Figure 27 by way of example, assume that the plane "p" selected at^step 
S30-4 is plane "a". Accordingly, the 3D points in the set Vp are the 3D points V1 , V2, V3 and V4. 

[0304] Assume also that the 3D point V2 vsas selected as the start vertex (u) at step S30-10. Accordingly, at 
step S30-16, polygon generator 86 selects either plane b or plane c since both of these planes are defined in the 
50 signature of the 3D point V2 together with the plane a. Assume that plane c is selected at step S30-16. 

[0305] Referring again to Figure 30, at step S30-18, polygon generator 86 removes the current vertex "w" (that 
is, 3D point V2 in the example being considered) from the set Vp, and at step S30-20 adds the vertex to the polygon 
data structure "s" generated at step S30-8 to define a vertex of the polygon "s". 

[0306] At step S30-22, polygon generator 86 determines the set "Vpq" of 3D points which remain in the set Vp and 
^ have the ID of plane "q" selected at step S30-1 6 in their signature. 

[0307] Referring again to the example in Figure 27, 3D point V3 has the signature (a, c, d} and is the only 



30 



I' 

dSined afkp S3S2 • "^^"'^'"^ ^^-^^^ ^"V 3D points in the set Vpq 

polygon have been processej no ISSI S SsftJ Xfe sef Vorfn thit^^r'"'""''' """" °' ^ - 

^ f * vvhich polygon generator 86 determines vvh^SV^rfanTaoSi s emaTr^ng ^n^^'fsey'v;'*""' *° ^''^ 

TsSL in^irSSI'S i^SrVe^are^IJLr^^^ *^ PO-vgonsUg r.^lane "p«, for example 
test at step S3§-24 Jdete.^in?S there a^^^ PotTsTthe set Vm'^S ^^^^^ ^'^^ 

p^eeds from step S30-e ^to lo-S^^w^^Sn-^dafa sSre^^enettTf^. JhVt^rthrpo^r 

point in the set Vpq reprSte fsD S J^V^^^^ number of points in the set Vpq is equal to 1. Each 3D 
and only one, Point'inThfsefvpq.The^rcSfexr^ current vertex V. Thus, there is one, 

more than one point in the set Vpq, then a case exSsuch a?th A™ '"flS"'® 27. However, if there is 

1020 and 1030 described above ® example in Figure 28 for the 3D points 1000, 1010, 



10 



IS 



^ SVset'Jtltep^l^^^^^^^^^^ ^-"'"'"9 »° ^« plane selected at step S30-16, the next 

W tolertex??'''''"' *° ^'^P ^» ^'•<='^ polygon generator 86 sets the current vertex pointer 



25 



30 



35 



40 



45 



50 



55 



seSS the p'Jln??r--^d'S]red infciraSr^^^ f '""l «"<^ ^t step S30-48. 

selected at step S30-16. T^us in the examote^ F^^ur^^^^ ^tl J^ f ^'^"^^ftrf* ^^0-4 and is not plane "q" 
planed is selected at step SSO-^S ^ ^ ^ signature of 3D point V3 is {a. c. d) and therefore 

detemiinedatstepS30-24thltthefe are7oX?fn ^o'^P'^'e v^*1en it is 

have been connected P^ ° accordingly all of the vertices of the polygon 

S;iL hasSers3c"ar;^at1?o^^„\L'^^^^^^^^^^^^^ °- Po'- set Vpq, a 

1030 lie in a line defined by the inS^ on of^^'ja ' ^ ^000, 1010, 1020, 

to detemnine >^ich 3D point in the set Vp^ is o^ Snnitld to th^3D no^n?? ^^""TT- ^'-^^""^ Processing 
the relative positions of the 3D point current^ beivTa c^nlwpr^^^ 

straight line on which they all lie currently being considered and the 3D points in the set Vpq along the 

KenlmberoT^DCS^^^fsX?''"' - - odd number or 

Sse the^'/D-S mS S^ontTd fn" pat 'l^irto^rc^iT^HT 1 ^'^"^^ ''^ 
example in Figure 28. 3D point 1000 is connected ?o ^^^ to fourth, etc. Thus, referring to the 

[0321] However, the number S 3D points In t!sexXuxeL7if^^^^^ ''T^'^'' *° point 1030. 

occur in vy*iich one of the 3D points lyina on the edoP T1 ""^^ ''^ o'^^ because a case can 

Kge o.''^^^J::-L'.^^^^^^ one of the vertices iying on 

having more than tvro vertices (that is he Itan^enlx V^^ ^^^^ processed is not a vertex lying on the edge 
tvvo vertices and the polygon is traversed in a lecS Xch has nSir'^ "^"^^^ °" ^^^'"9 "^o^e than 

determlnea ,o contain m,ee 3D pJ° W„X SSo« To^'^S'SST f'^. 5 
o. »e 3D po.« ,0,0, or ,03? - SerSifveTj^.^^^ r.^.r^SSrlZ 



current vertex "w" it will be removed from the set Vp at step S30-18, resulting in tw^ points lying on the 
Intersection of planes 910 and 1040 remaining In the set Vpq tested at steps S30-24 and S30-26 (that is, an even 
number of points). 

[0324] Consequently, if It is determined at step S30-30 that the number of points in the set Vpq is odd, polygon 
generator 86 defines a set "Vline" of the 3D points lying on the intersection of planes "p" and "q" such that the 
set comprises the current vertex 'W and the 3D points in the set Vpq. 

[0325] On the other hand, If it is determined at step S30-30 that the number of points in the set Vpq Is even, 
^ then at step S30-32. polygon generator 86 defines a set "Vline" of 3D points lying on the Intersection of planes p 
and q such that the set comprises the current vertex 'W, the start vertex "u" and the 3D points in the set Vpq. 
[0326] Following step S30-32 or step S30-34, processing proceeds to step S30-36, at which polygon generator 86 
assigns a rank to each 3D point in the set Vline from rank 0 to rank n-1 (where n Is the number of points In the set 
Vline) in dependence upon their relative positions along the line defined by the intersection of planes "p" and "q". 
1Q More particularly, one of the end points on the intersection (it does not matter which) is assigned rank 0 and the 
remaining points In the set Vline are assigned rank 1, 2, etc as their distance from the point of rank 0 increases. 
Thus, referring to the example shown in Figure 28, if 3D point 1000 is assigned rank 0, then 3D point 1010 is 
assigned rank 1, 3D point 1020 is assigned rank 2 and 3D point 1030 is assigned rank 3. 

[0327] At step S30-38, polygon generator 86 determines whether the rank of the 

current vertex "w" is even or odd. 
15 [0328] If it is determined at step S30-38 that the rank of the current vertex "w" is even, then at step S30-40, 
polygon generator 86 sets the next vertex "x" of the polygon "s" (that is the vertex connected to the current vertex) 
to be the vertex "v" In the set Vpq such that: 

rank (v) = rank (v\^ + 1 (3) 

20 [0329] On the other hand, If It Is determined at step S30-38 that the rank of the current vertex "w" is odd, 
then at step S30-42, polygon generator 86 sets the next vertex "x" and the polygon "s" to be vertex "v" In the set 
Vpq such that: 

rank (v) = rank (w) - 1 (4) 

25 [0330] Processing then proceeds to step S30-44. 

[0331] As described above, at step S30-44, polygon generator 86 sets the current vertex pointer 'W to vertex 
"x" and at step S30-46 reads the signature of the new current vertex. 

[0332] . At step S30-48, polygon generator 86 selects the plane "r" defined in the signature read at step S30-46 
which is not plane "p" selected at step S30-4 and is not plane "q" selected at step S30-16. 
[0333] After step S30-48, processing returns to step S30-18. 

[0334] Returning again to step S30-6, when it is determined that there are no more 3D points in the set Vp 
(because all of the 3D points have been removed on previous Iterations of processing at step S30-18), then 
connections have been detemnined for all of the 3D points lying on the plane "p" selected at step S30-4 and 
processing proceeds to step S30-50. 

[0335] At step S30-50, polygon generator 86 determines whether there is more than one polygon lying in the plane 
35 "p". More particularly, in this embodiment, polygon generator 86 determines whether more than one polygori data 
structure "s" has been generated for the plane "p" (a new polygon data structure "s" being generated for^ each 
polygon lying in the plane "p" at step S30-8). 

[0336] If it is determined at step S30-50 that there is more than one polygon lying in the plane "p" processing 
proceeds to step S30-52, at which polygon generator 86 tests each polygon lying in the plane "p" to detemnine whether 
it contains any of the other polygons lying In the plane "p". 
^0 [0337] At step S30-54, polygon generator 86 determines whether each polygon lying in the plane "p" represents 
part of the surface of the subject object 210 or a hole therein. More particularly, if a polygon is not contained 
with any other polygons, then it represents a surface of the subject object 210. For a case in which one, and only 
one, polygon lies inside another polygon, then the larger polygon represents the surface of the subject object 210 
and the smaller polygon which is contained in the larger polygon represents a hole in the surface. For cases in 
wtijch there Is more than one polygon contained in a polygon, polygon generator 86 creates a hierarchy such that each 
parent polygon contains its children polygons. Polygon generator 86 then identifies as surface of the subject object 
210 each polygon in the odd generations in the hierarchy and identifies as holes 
those polygons in the even generations. 

[0338] On the other hand, if it is determined at step S30-50 that there is only one polygon in plane "p", steps 
S30-52 and S30-54 are omitted. 
50 [0339] At step S30-56, polygon generator 86 determines whether there is another plane identified at step S23-2 
to be processed. 

[0340] If it Is determined at step S30-56 that another plane remains to be processed, then processing retums to 
step S30-4. 

[0341] Steps S30-4 to S30-56 are repeated until each plane has been processed in the way described above. 
[0342] As a result of this processing, polygon generator 86 calculates a polygon mesh representing the surface 
of subject object 210. 
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user input devices 2006, and also provides control and processing for the other functional units. 

[0358] Memory 2030 is provided for use by central controller 2020 and the other functional units. 

[0359] Input data store 2040 stores Input data input to the processing apparatus 2002, for example as data 

stored on a storage device, such as disk 2042, as a signal 2044 transmitted to the processing apparatus 2002, or 

using a user input device 2006. The Input data defines one or more 3D computer models, each made up of planar 

polygons In a conventional manner. 

[0360] Component object modeller 2050 is arranged to process the Input data stored In input data store 2040 to 
amend the input 3D computer models to change their shape, appearance etc in accordance with user Instructions. In 
addition, component object modeller 2050 is also arranged to generate data defining new 3D computer models made 
up of planar polygons In accordance with Instructions Input by a user. 

[0361] Model positioner 2060 is arranged to process instructions input by a user using a user input device 2006 
to position the objects defined In the Input data stored In input data store 2040 and/or defined by the data generated by 
component object modeller 2050 relative to each other in a common coordinate space, so as to define components of a 
compound object. 

[0362] Compound object modeller 2070 comprises a 3D point calculator 2080 and a polygon generator 2090. 
[0363] 3D point calculator 2080 is arranged to process the data defining the component 3D computer models and 
their relative positions to calculate the 3D positions of the intersections of planar polygons making up the 
component models (which intersections define potential vertices of the compound object) and to test the 
Intersections to determine which ones represent actual values of the compound object. 

[0364] Polygon generator 2090 is arranged to connect the 3D points generated by 3D point calculator 2080 to form 
a polygon mesh representing the surface of the compound object. 

[0365] Display processor 2100, under the control of central controller 2020, is arranged to display images and 
Instructions to a user via display device 2004 during the processing by component object modeller 2050, model 
positioner 2060 and compound object modeller 2070. In particular, display processor 2100 is arranged to display 
images of the component and compound computer models from user-selected viewpoints. 

[0366] Output data store 2110 is arranged to store the data defining the compound object generated by compound 
object modeller 2070. Central controller 2020 is arranged to control the output of data from output data store 2110, 
for example as data on a storage device, such as disk 2120 and/or as a signal 2130. 

[0367] Figure 32 shows the processing operations performed by processing apparatus 2002 in this embodiment. 
[0368] Referring to Figure 32, at step S32-2, component object modeller 2050 acts on instructions input by a 
user via a user input device 2006 to generate data defining a respective 3D computer model for each of a plurality 
of component objects. This processing is performed in a conventional manner, for example as In a conventional 
computer-aided design (CAD) apparatus. 

[0369] Also at step S32-2 display processor 2100 generates image data for display on display device 2004 to 
display the component objects to the user. In this embodiment, the 3D computer models of the component objects are 
generated In an interactive way, that Is, the effect of the input of each Instruction from the user on the formation 
of a 3D computer model Is displayed on display device 2004 and the user inputs further instructions to complete the 
3D computer model in response to the successive display of input effects. 

[0370] An example of the result of the processing at step S32-2 is shown In Figure 33a, in which three 3D 
component objects 2200, 2210, 2220 defined by data generated by component object modeller 2050 are illustrated. 
[0371] Component object 2200 comprises a bottom face polygon 2230, a top face polygon 2240, and side face, 
polygons 2250, 2260, 2270 and 2280, 

[0372] Component object 2210 comprises a top face polygon 2290, a bottom face polygon 2300, and side face 
polygons 2310, 2320 and 2330. 

[0373] Component object 2220 is made up of polygons 2340, 2350, 2360, 2370, 2380, 2390, 2400 and 2410. 
[0374] The component objects shown In Figure 33a will be referred to below to illustrate subsequent processing 
performed in this embodiment. 

[0375] Instead of component object modeller 2050 generating data to define every component object at step S32- 
2, one or more of the component objects may be defined by data input to processing apparatus 2002 and stored in 
input data store 2040. Such input data may be generated, for example, using the processing described in the first 
embodiment, or by laser-scanning one or more objects etc. 

[0376] In addition or instead, the data stored in input store 2040 for one or more of the 3D computer models may 

be amended by component object modeller 2050 in accordance with instructions input by a user via a user input device 

2006 to edit the 3D computer model and hence change the shape, appearance etc of the component object. 

[0377] Referring again to Figure 32, at step S32-4, model positioner 2060 acts on instructions input by a user 

via a user Input device 2006 to position the component objects generated at step S32-2 (or defined in the data 

stored in input data store 2040) in a common coordinate space as parts of a compound object. In addition, display 

processor 2100, under control of central controller 2020, generates Image data for display on display device 2004 to 

display images of the component objects as they are positioned In the common coordinate space. 

[0378] Figure 33b shows an example of the result of the processing at step S32-4, in which the component objects 

2200, 2210 and 2220 of Figure 33a are positioned relative to each other as parts of a compound object. 

[0379] Referring to Figure 33b, component object 2210 is positioned so that it penetrates component object 2200 

through the polygon 2230 (that is, the vertices 2420, 2430 and 2440 defining the polygon 2290 of component object 

2210 are inside component object 2200). 
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least one of the original 3D objects). 

[0394] In this embodiment, when a 3D point is calculated, it is subsequently tested to determine whether it 
represents a vertex of the compound object. This is because a 3D point is calculated when a 3D volume is identified 
which contains three planar polygons. However, the polygons may not actually intersect at a 3D point within the 
volume. In addition, the 3D point is calculated by calculating the intersection of the three respective planes in 
which the polygons lie, with the result that the calculated 3D point mat not lie within one or more of the polygons. 
Accordingly, in this embodiment, a test is carried out to determine whether the calculated 3D point is within the 
^ identified volume and is within all three of the polygons. 

[0395] The processing performed in this embodiment to calculate 3D points representing vertices of the compound 
object will now be described in detail. 

[0396] Figure 34 shows the processing operations perfonned by 3D point calculator 2080 at step S32-6 to 
calculate 3D points defining vertices of the compound object. 
10 [0397] Referring to Figure 34, at step S34-2, 3D point calculator 2080 defines a volume in 3D space in which all 
3D points representing vertices of the compound object will lie. 

[0398] More particularly, referring to Figure 35, in this embodiment, 3D point calculator 2080 defines an axis- 
aligned bounding cuboid 2500 around the component objects 2200, 2210 and 2220 as the volume containing the 
compound object vertices at step S34-2. The bounding cuboid is defined in a conventional manner, and accordingly the 
processing will not be described here. 
15 [0399] Referring again to Figure 34, at step S34-4, 3D point calculator 2080 subdivides the 3D volume defined at 
step S34-2 into a plurality of smaller, child volumes, and adds the child volumes to the top of a storage stack 
defined in memory. 

[0400] More particularly, referring to Figure 36, in this embodiment the subdivision of the volume at step S34-4 
comprises the binary subdivision of the bounding cuboid 2500 to generate eight new child volumes 2510, 2520, 2530, 
20 2540, 2550, 2560, 2570 and 2580. 

[0401] Referring again to Figure 34, at step S34-6, 3D point calculator 2080 defines a respective polyhedron for 
each component object which contains the component object as a hole. 

[0402] This processing and Its effect will be described for subject object 2200 with reference to Figure 37 (the 
processing and its effect being the same for each of the other component objects). 

[0403] Referring to Figure 37, the processing at step S34-6 defines a polyhedron 2600 having the fomn of a 
25 cuboid with a hole 2610 therein having the same coordinates as the original component object 2200 (referred to 
hereinafter as a "component object hole"), 

[0404] Thus, the polygons of the original component object now define a component object hole in the polyhedron 
2600. 

[0405] More particularly, 3D point calculator 2080 performs the processing at step S34-6 by defining cuboid 2600 
to have faces each of which lies a small distance outside the corresponding face of the bounding cuboid 2500 
previously defined at step S34-2. Thus, cuboid 2600 has the same shape as bounding cuboid 2500 but is slightly 
larger in size. In this way, cuboid 2600 encloses all points on the compound object to be generated. 
[0406] As a result of performing step S34-6, processing to calculate the union of the volumes occupied by 
the component objects 2200, 2210, 2220 now becomes processing to calculate the union of the component object 
holes in the polyhedra. 

35 [0407] Referring again to Figure 34, at step S34-8, 3D point calculator 2080 pops the next volume off the top of 
the stack created at step S34-4 (this being the first volume the first time step S34-8 is performed), and at step 
S34-1 0, determines the state of the volume popped off the stack. 

[0408] In the processing at step S34-10, 3D point calculator 2080 determines whether the volume popped off the 
stack at step S34-8 cannot contain any 3D point representing a vertex of the compound object (and hence can be 
discarded), whether the volume needs to be subdivided because it contains a sufficient number of polygons to define 
more than one 3D point which may represent a vertex of the compound object, or whether the volume contains only 
enough polygons to define just a single 3D point which may represent a vertex of the compound object, in which case 
the position of the 3D point can be calculated and tested. 

[0409] Figure 38 shows the processing operations perfonned by 3D point calculator 2080 at step S34-10. 
[0410] Referring to Figure 38, at steps S38-2 to S38-10, 3D point calculator 2080 performs processing to 
45 determine which polygons defining the component object holes intersect the volume popped off the stack at step S34- 
8. This processing is performed in a computationally efficient way by testing only the polygons which intersect the 
parent volume (that is, the volume divided to create the volume currently being processed) for intersections* with 
the current volume, and storing a list of the polygons which do intersect the current volume for use when a child 
volume of the current volume is processed. 

[0411] More particularly, at step S38-2, 3D point calculator 2080 considers the list of polygons which intersect 
50 the parent volume (this being all of the polygons defining the component object holes when the parent volume is the 
bounding cuboid 2500 defined at step S34-2). 

[0412] At step S38-4, 3D point calculator 2080 considers the next polygon on the list read at step S38-2 (this 
being the first polygon the first time step S38-4 is performed), and at step S38-6 determines whether the polygon 
intersects the current volume. The processing performed in this embodiment to test whether a polygon intersects the 
current volume is carried out in a conventional way, for example as described in Graphics Gems V pages 375-379, 
edited by Alan W. Paeth and published by Morgan Kaufmann, ISBN 0-12-543455-3. 
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intersect the polyhedron 2600 at an even number of places. In the example of Figure 41b, the ray 2690 intersects the 
polyhedron 2600 at two places, namely points 2700 and 2710. However, the ray 2690 may intersect the polyhedron 
2600 at a higher even number of places depending upon the shape of the subject object hole 2610. 
[0429] Referring again to Figure 39, if it is determined at step S39-6 that the number of intersections of the 
ray projected at step S39-4 with the polyhedron is odd, then at step S39-8, 3D point calculator 2080 sets a flag to 
Indicate that the volume popped off the stack at step S34-8 is enclosed by the polyhedron. 

[0430] On the other hand, if it Is determined at step S39-6 that the number of Intersections of the ray 
projected at step S39-4 with the polyhedron Is even, then at step S39-10, 3D point calculator 2080 sets a flag to 
indicate that the volume popped off the stack at step S39-8 is disjoint from the polyhedron (that Is, the volume Is 

wholly inside the subject object hole of the polyhedron). 

[0431] Referring again to Figure 38, at step S38-20, 3D point calculator 2080 reads the flag set at step S39-8 
or^step S39-1 0 to determine whether the current volume is enclosed by, or is disjoint from, the polyhedron. 
10 [0432] If it is determined at step S38-20 that the volume popped off the stack at step S34-8 is disjoint from the 
polyhedron, then at step S38-22, 3D point calculator 2080 sets a flag to indicate that the state of the volume 
popped off the stack at step S34-8 has been determined to be "discard" because the volume lies completely inside one 
of the component object holes (and therefore completely inside one of the original 3D component objects) and hence 
cannot contain a 3D point representing a vertex of the compound object. 

[0433] After step S38-22 has been performed, processing returns to step S34-12 in Figure 34. This Is because It 
Is not necessary to determine the relationship between the current volume and any further polyhedra because only one 
polyhedron which Is disjoint from the current volume needs to be identified for 3D point calculator 2080 to 
determine that the current volume cannot contain a 3D point representing a vertex of the compound object. 
[0434] On the other hand, if It Is determined at step S38-20 that the volume popped off the stack at step S34-8 
Is enclosed by the polyhedron, then at step S38-24, 3D point calculator 2080 deletes the polyhedron from the list of 
2Q active polyhedra for the current volume, so that the polyhedron is not tested to detemrilne whether it encloses any child 
volumes of the current volume. 

[0435] Processing then proceeds to step S38-26, at which 3D point calculator 2080 determines whether there is 
another polyhedron on the list copied at step S38-12 containing no polygons which intersect the current volume. 
Steps S38-16 to S38-26 are repeated until each such polyhedron has been processed in the way described above, or 
until it is determined at step S38-20 that the current volume is disjoint from one of the polyhedra (in which, case 
25 the state of the volume is determined to be "discard" at step S38-22 and processing returns to step S34-1 2). 

[0436] When it is determined at step S38-14 that there are no polyhedra on the list copied at step S38-12 having 
no polygons which intersect the current volume, or when it is determined at step S38-26 that all such polygons have 
been processed and found to enclose the current volume, processing proceeds to step S38-28. 

[0437] At step S38-28, 3D point calculator 2080 reads the list generated at step 838-8 defining the polygons 
which intersect the current volume. 

[0438] At step S38-30, 3D point calculator 2080 detemriines whether the number of polygons on the list read at 
step S38-28 is 3, more than 3 or less than 3. 

[0439] If it is determined at step S38-30 that the number of polygons which intersect the current volume is less 
than 3, then, at step S38-32, 3D point calculator 2080 sets a flag to indicate that the state of the volume popped 
off the stack at step S34-8 has been determined to be "discard" because the volume does not contain a sufficient 
35 number of polygons to intersect and generate a 3D point representing a vertex of the compound object. Processing 
then returns to step S34-12 in Figure 34. 

[0440] If it is determined at step S38-30 that the number of polygons which intersect the current volume is 3, 
then, at step S38-34, 3D point calculator 2080 sets a flag to indicate that the state of the volume popped off the 
stack at step S34-8 has been determined to be "calculate vertex" because the volume contains the correct number of 
polygons to intersect and define a single 3D point which may represent a vertex of the compound object. Processing 
then returns to step S34-12 in Figure 34. 

[0441] If It is determined at step S38-30 that the number of polygons which intersect the current volume Is 
greater than 3, then, at step S38-36, 3D point calculator 2080 sets a flag to Indicate that the state of the volume 
popped off the stack at step S34-8 has been determined to be "subdivide" because tt contains more than the required 
number of polygons (that Is 3) to intersect and generate a single 3D point which may represent a vertex of the 
coTT»pound object. Processing then returns to step S34-1 2 In Figure 34. 

[0442] As a result of the processing described above with reference to Figure 38, 3D point calculator 2080 has 
determined the state of the volume popped off the stack at step S34-8 and has set a flag in dependence upon the 
determined state indicating what subsequent processing Is to be performed on the volume. 

[0443] Referring again to Figure 34, after performing processing as described above to determine the state of 
the volume, at step S34-1 2, 3D point calculator 2080 determines whether the state is "calculate vertex". 
50 [0444] If it is determined at step S34-12 that a vertex is to be calculated, processing proceeds to step S34-14 
to calculate the vertex, otherwise step S34-14 is omitted. 

[0445] Figure 42 shows the processing operations performed by 3D point calculator 2080 at step S34-14 to 
calculate a vertex. 

[0446] Referring to Figure 42, at step S42-2, 3D point calculator 2080 calculates the 3D position of the point 
at which the three planes containing the three respective polygons Intersect (this processing being performed using 
a conventional plane intersection technique). 
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SunderstanSg. HovJ^veT' atS"4qf;:The^'ro7^^^^^^^ °1h'' ^^°vvn by dotted liles to 

been calculated, and aSngV pro?e?s1?q Tp^aX 2002 ?.;l^r,n^°""^'^'^^ '"^^ vertices have not yet 

will now be described processing apparatus 2002 performs processing to calculate the connections, as 

representing the surface of the compound object '"ereDetween and therefore to generate a polygon mesh 

£S,edby'^oVgon7eneLr8°6"^^^^^^^^^ ^^^-^ ^"'^ as the processing 
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of the compound object ^ signature for each cateulated 3D point representing a vertex 

&s calcl^d'S sr^oim^alcSftS^S S^" sTnd'sZ^"^'^ " ^ "^^'^^^^ ^vhich a 3D 
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'SL ID htl t^att^ne^SSS^S^^^^^^^ '° -^^on Llected'it step S44-2 unless 

Krte^of the^^i^Su'^bje'^^^^^^^^^^ -c*- Po'vgon used to calculate a 

the way described above repeated until each such polygon has been processed in 

..e IDs o. ,ha poison, *oh .e«, „ .he'sD PowXSSg'^rX^.r.ht^iviSrsS 



544- 4). 

[0466] At step S44-12, polygon generator 2090 determines whether there is another calculated vertex of the 
compound object, and steps S44-8 to S44-12 are repeated until each vertex has been processed in the way described 
above. 

[0467] As a result of performing the processing at step S32-8, polygon generator 2090 has generated a mesh of 
triangular polygons representing the surface of the compound object. 

[0468] Referring again to Figure 32, at step S32-10, central controller 2020 outputs the data defining the 3D 
computer model of the compound object from the output data store 2110, for example as data stored on a storage 
device, such as disk 2120, or as a signal 2130 (Figure 31). In addition, or instead, central controller 2020 causes 
display processor 2100 to display an Image of the 3D computer model of the compound object on display device 2004 
in accordance with a viewpoint Input by a user, for example using a user Input device 2006. 

10 Third Embodiment 

[0469] A third embodiment of the present Invention will now be described. 

[0470] The components of the third embodiment and the processing operations performed thereby are the same 
as those in the first embodiment, with the exception of the processing operations performed by surface modeller 80 
at step S4-16 In Figure 4. which are different In the third embodiment. 
[0471] These differences will now be described. 

[0472] Figure 45 shows the processing operations perfomned by surface modeller 80 in the third embodiment at 
step S4-16 in Figure 4. 

[0473] Referring to Figure 45, at step S45-2, silhouette approximator 82 processes the segmented image data 
previously generated by image data segmenter 60 at step S4-8 and image segmentation editor 70 at step S4-12 in 
2Q Figure 4 to approximate the silhouette of the subject object in each input Image with straight lines. This 
processing is the same as the processing performed by silhouette approximator 82 In the first embodiment at step S8- 
2 in Figure 8, and results in each silhouette being approximated by connected straight lines as illustrated in the 
example of Figure 9. 

[0474] At step S45-4, 3D point calculator 84 projects each straight line of each silhouette into three- 
dimensional space to generate an Infinite planar triangle having sides passing through the end points of the 
25 Straight line and meeting at the position of the focal point of the camera 230 when the image containing the 
straight line was recorded (the focal point position of each Image having previously been calculated by camera 
calculator 50 at step S4-6). 

[0475] Referring back to Figure 10, the processing performed at step S45-4 generates a polyhedron for each Input 
image comprising an infinite cone 520 made up of triangular planar faces (one for each straight tine segment of the 
silhouette) which touches the surface of the subject object 210 at (as yet unknown) points in the three-dimensional 
space and which has its appex at the focal point position of the camera 230. The cross-section of each cone is 
therefore the same shape as the silhouette from which it was generated. 

[0476] 3D point calculator 84 assigns a unique identification (ID) to each planar face in each polyhedron. 
[0477] At steps S45-6 to S45-18, 3D point calculator 84 performs processing to calculate the points of 
Intersection of the planar faces from the polyhedra generated at step S45-4 and to test the calculated points of 
35 intersection to determine whether they represent points on the surface of the subject object 210. 

[0478] More particularly, at step S45-6, 3D point calculator 84 considers all of the planar faces from all of 
the polyhedra generated at step S45-4, and considers the next set of three of these planar faces which contains 
faces from at least two polyhedra (this being the first such set the first time step S45-6 Is performed). 
[0479] At step S45-8, 3D point calculator 84 calculates the point of intersection of the three planar faces 
considered at step S45-6. More particularly, In this embodiment, the point of Intersection is calculated by 
calculating the point at which the three respective planes containing the three - planar faces meet using a 
conventional plane intersection algorithm, 

[0480] At step S45-10, 3D point calculator 84 determines whether the point of intersection calculated at step 

545- 8 is within all of the polyhedra generated at step S45-4. 

[0481] If it is determined at step S45-10 that the point is outside at least one polyhedron, then at step S45-12, 
the -ipoint is discarded because it cannot represent a point on the surface of the subject object 210. 
[0482] On the other hand, if it is determined at step S45-10 that the point is within all of the polyhedra, then 
at step S45-14, 3D point calculator 84 determines whether the point Is within all of the three planar faces 
considered at step S45-6 (It being noted that the point may be outside one or more of these planar faces because the 
point was calculated by calculating the point of Intersection of the planes containing the faces and not by 
calculating the point of intersection of the planar faces themselves), 
50 [0483] If it Is determined at step S45-14 that the point Is outside at least one of the planar faces, then at 
step S45-12 the point Is discarded because it cannot represent a point on the surface of the subject object 210. 
[0484] On the other hand, If it is determined at step S45-14 that the point is within all three of the planar 
faces, then at step S45-16, the point is retained as a point on the surface of the subject object 210. In addition, 
the point is assigned a "signature" comprising a triple fomied from the three respective identifications of the planar 
faces which meet at the point (these identifications being the identifications previously assigned to the planar 
faces at step S45-4). 



f?,!5 ^ • , , calculator 84 determines vUiether there is another set of three planar faces 
containing faces from at least two polyhedra. Steps S45-6 to S45-18 are repeated until each respective set of three 
planar faces from at least tw) polyhedra has been processed In the v«y described above ^^^pective set or tnree 
i„tl?S^i«„ ^ ^ °* Pf^Of^'ng this processing, 3D point calculator 84 has calculated and tested the point of 
^mn^f^^.f''11P°^^^'® combination of three planar faces from at least two polyhedra, to generate a 3D 
SSrt obS?llo ° '^"^ comprising points in 3D space representing points on the surface of the 
^ [04871 At step S45-20, polygon generator 86 connects the 3D points retained at step S45-16 using the sianatures 
fpnrfcf^tinn 5f ^°'T '° ^'^h points should be connected together, thereby generating a polygon mesh 
nni^normfch '"^^''^ o'jhe subject object 210. Accordingly, the processing to connect the 3D points to generate a 
polygon mesh is perfomied after the 3D points resulting from the intersections of all of ths nnivhsHra generated at 
step t>4tj-4 nave been calculated and tested. " generaieo ai 

\^t^tLr.J^f processing performed by polygon generator 86 at step S45-20 in the third embodiment is the same as 
that perfomied at step S8-8 in the first embodiment, with the exception that steps S25-10 to S25-14 in Fiaure 25 are 
Tniff, ""^ A.'u*®!^"^ embodiment because this processing Is instead performed at step S45-16 in Figure 45 
[04891 Although more computationally expensive than the processing perfomried in the first embodiment the 
processing perfomied in the third embodiment to generate a 3D computer model of the subject object 210 still 
IS in2Z?® '®ss processing resources and/or time than the prior art methods of generating a 3D computer model 
modifications can be made to the embodiments described above within the scope of claims 
[04911 For example, in the first and third embodiments described above, the input image data comprises "still" 
images of the subject object 210 recorded by camera 230. However, the input images may comprise frames of image 
aata trom a video camera. 

[0492] In the first and third embodiments described above, at step S4-4, data input by a user defining the 
20 |nt"f^sic parameters of the camera Is stored. However, instead, default values may be assumed for some or all of 
the intrinsic camera parameters, or processing may be performed to calculate the intrinsic parameter values in a 
conventional manner, for example as described in "Euclidean Reconstruction From Uncalibrated Views" by Hartley in 
Applications of Invanance in Computer Vision, Mundy, Zisserman and Forsyth eds, pages 237-256 Azores 1993 
[0493] In the first and third embodiments described above, image data from an input image relating to the 
subject object 210 is segmented from the image data relating to the background as described above with reference to 
Figure 5. However, other conventional segmentation methods may be used instead. For example, a segmentation 
method may be used in which a single RGB value representative of the colour of the photographic mat 34 and 
background (or just the background) Is stored and each pixel In an Input image Is processed to determine whether the 
threshofd" ® ^ between the RGB background value and the RGB pixel value Is less than a specified 

30 ^^^^V ^'^^ embodiments described above, camera calculator 50 calculates the relative 

posrtions and onentations of the input images by calculating the position and orientation of the camera 230 
relative to the mat 34 for each input Image (described above with reference to step S4-6). However, the way in which 
the imaging positions and orientations are calculated Is not important. For example, the mat 34 may be discarded and 
instead, the input images may be generated using a plurality of cameras surrounding the subject object 210 with 
each camera having a known, fixed position and orientation which is input to the processing apparatus 2 

55 Alternatively, the photographic mat 34 may be discarded and the subject object 210 may be mounted on a turntable 
. having position encoders which input the precise rotation of the turntable to processing apparatus 2 Knowing the 
fixed position of camera 230 and the turntable angle, processing apparatus 2 can then calculate the relative 
position and orientation of each image of the subject object 210. 

[0495] In the first embodiment deschbed above, a flag "eitherSubdivideOrDiscard" is set during the processing 
^ performed at step Si 3-8. However, it is not essential to set this flag because the processing will still work as a 
w result of the processing to cause the "np" counter to overflow (that Is, exceed three in value in the first 

embodiment) at steps Si 5-26 and Si 5-46. 

[0496] In the first embodiment described above, at step 815-16, processing apparatus 2 generates a set of 
candidate edges" which may Intersect the projected volume. Subsequent tests are then restricted to edges lying in 
the set of candidate edges. However, step Si 5-1 6 (and also step S8-4 which is performed to facilitate the processing 
4g of step S15-16) may be omitted, and subsequent tests carried out on all edges. However, this is likely to increase 
significantly the amount of processing operations needed to be performed. 

[0497] In the first embodiment, instead of performing the processing described above with reference to step S13- 
2 to define a 3D volume In which all calculated points on the object surface will lie, processing may be carried out 
to define the 3D volume in different ways. For example, an Initial start volume may be defined to be an arbitrary, large 
cube. A cube is then either added to the Initial volume (if ail of its corners are In front of all of the cameras) 

50 or is discarded (if all of Its corners are behind one of the cameras or if the cube Is smaller than some 
predetermined size) or is subdivided (if neither of these conditions apply). Subdivided cubes are then processed In 
the same way. In this way, the 3D volume defined at step S13-2 will consists of the union of a set of cubes and is 
guaranteed to lie entirely in front of ail of the cameras and to be the largest such region up to the tolerance 
given by the minimum cube size. 

^ [0498] In the first embodiment described above, the volume of 3D space is repeatedly sub-divided until a 
• sufficiently small volume Is obtained that It contains only a single vertex. However, instead, sub-division of a 3D 



volume may be stopped when the volume contains a predetermined number of vertices greater than or equal to two. 
Each of the plurality of vertices would be calculated and tested to determine whether it represents an actual vertex 
of the subject object in the same way that each single vertex is calculated and tested in the first embodiment. 
[0499] The processing operations described for the first and third embodiments can be extended in an obvious 
manner to the case where a silhouette generated at step S8-2 contains polygons representing holes (and also islands 
within holes etc). 

[0500] In the second embodiment described above, at steps S38-30 and S38-36, the state of a volume is set to 
"sub-divide" If the number of polygons that intersects the current volume is greater than 3. However, instead, if 
there are more than 3 but less than or equal to a predetermined number (for example 5) of polygons intersecting the 
current volume, the state of the volume may be set to "calculate multiple vertices". In subsequent processing, 
processing apparatus 2002 vw3uld then calculate and test each vertex defined by a subset of three of the polygons 
intersecting the volume. A volume would therefore only be subdivided if there were more than the predetermined 
maximum number of polygons intersecting the volume. In this way, a volume could add more than one vertex to the 
mode! of the compound object, thereby saving processing to further subdivide the volume (at the cost of having more 
candidate vertices to compute and test). 

[0501] When subdividing a 3D volume popped off the stack in the first embodiment at step S13-18, and in the 
second embodiment at steps S34-4 and S34-20, the subdivision carried out comprises the binary subdivision of the 
volume to generate eight new child volumes. However, different types of subdivision are, of course, possible. 
[0502] In the third embodiment described above, at step S45-8, the point of intersection of three planar faces 
is calculated by calculating the point at which the three respective infinite planes containing the faces intersect. 
Tests are then carried out at steps S45-10 and S45-14 to determine whether the calculated point is within all of the 
polyhedra and is within all of the three planar faces. However, instead, at step S45-8, the point of intersection of 
the three planar faces themselves may be calculated (instead of the planes in which they lie), in which case steps 
S45-10 and S45-14 may be omitted. 

[0503] The third embodiment above describes a modification to the processing in the first embodiment which 
generates a computer model of the surface of the subject object. The processing in the second embodiment to 
calculate and test the intersections of the polyhedra and to connect the resulting points to generate a surface 
model could be modified in a corresponding way. 

[0504] In the first and second embodiments described above, the initial volume defined so that it will contain 
all of the points to be calculated on the surface of the object (defined at step S13-2 in the first embodiment, and 
step S34-2 in the second embodiment) is repeatedly subdivided into smaller volumes until sufficiently small volumes 
are generated that each can contain only a predetermined number of 3D points on the surface of the object When such 
smaller volumes are reached through the sub-division process, the 3D points are calculated and tested. However, 
while this is a particularly efficient \A©y of calculating the points representing points on the surface of the 
object, other methods are possible. For example, the initial volume which will contain all of the calculated 3D 
points may be divided into a plurality of parts without reference to the positions of the planar polygons making up 
the polyhedra and 3D point calculation may be carried out without further subdivision of the parts. More 
particularly, the volume may be divided into a plurality of parts (for example of the same shape and volume) and 
then each part tested against the polyhedra to detemiine whether it is v\^olly outside at least one of the polyhedra. 
If the volume part is outside at least one polyhedron, then the volume part is discarded. On the other hand, if the 
volume is at least partially within all of the polyhedra, the 3D points in the volume part at which planar polygons 
in the polyhedra intersect are calculated. In this way, each volume part is discarded or 3D points are calculated, 
but no further subdivision occurs. To calculate the 3D points in a volume part, each combination of three planar 
polygons is considered and the intersection of these polygons is calculated and tested to determined whether it is 
within the volume part. This increases the number of intersections which need to be calculated and tested compared 
to the first and second embodiments described above, but is still an efficient way of generating the 3D computer 
model because volume parts can be discarded if they lie outside every polyhedra, and accordingly, not all possible 
intersections of the planar polygons making up the polyhedra have to be calculated and tested. 

[0505] In all of the embodiments described above, processing is performed by a computer using processing 
routines defined by programming instructions. However, some, or all, of the processing could, of course, be 
performed using hardvsare. 

Claims 

1. A method of processing data defining at least three polyhedra, each respective polyhedron comprising a plurality 
of planar faces in a three-dimensional space defining a volume within which at least part of a subject object 
lies, to generate a 3D computer model of the subject object comprising points in the three-dimensional space 
representing points on the surface of the subject object connected to form a polygon mesh, the method comprising: 

calculating points in the three-dimensional space representing points on the surface of the subject object 
by calculating the respective points at which a predetermined number of planar faces from the polyhedra 

intersect; 

assigning an identification to each calculated point representing a point on the subject object in 
dependence upon the planar faces which meet at the point; and 



after calculating the points representing points on the subject object and assigning identifications thereto, 
connecting the points In dependence upon the assigned identifications to generate a 3D computer model 
comprising a polygon mesh representing the surface of the subject object. 



2- A method according to claim 1, wherein, in the step of calculating points representing points on the subject 
object, tests are carried out to identify which planar faces cannot intersect at a point on the subject object by 
testing volumes of the three-dimensional space to identify volumes which do not contain any part of the subject 
object and the planar faces intersecting the identified volumes, and wherein points of intersection are not 
calculated for the planar faces which cannot intersect at a point on the subject object. 

3. A method according to claim 2, wherein the step of calculating points In the three-dimensional space 
representing points on the surface of the subject object comprises: 

defining a volume of the three-dimensional space in which all points on the subject object will lie; 

dividing the volume Into a plurality of volume parts; 

testing each volume part against the polyhedra, and: 

discarding the volume part if it is wholly outside at least one of the polyhedra; 

if the volume part Is at least partially within all of the polyhedra, processing the data defining the 
planar surfaces which intersect the volume part to calculate the 3D points in the volume part at which a 
predetermined number of the planar surfaces meet. 



4- A method of processing data defining a plurality of polyhedra, each respective polyhedron comprising a plurality 
of planar surfaces in a three-dimensional space defining a volume within which at least part of a subject object 
lies, to generate a 3D computer model of the subject object comprising points in the three-dimensional space 
representing points on the surface of the subject object, the method comprising: 

defining a volume of the three-dimensional space in which all points in the 3D computer model will lie; 

dividing the volume into a plurality of volume parts; 

testing each volume part against the polyhedra, and: 

discarding the volume part if it is wholly outside at least one of the polyhedra; 

if the volume part Is at least partially within all of the polyhedra, processing the data defining the 
planar surfaces which intersect the volume part to calculate the 3D points in the volume part at which a 
predetermined number of the planar surfaces meet. 



5. A method according to claim 3 or claim 4, wherein the steps of dividing the volume into volume parts and testing 
the volume parts comprise testing the volume parts against the polyhedra, and: 

discarding a volume part if it is wholly outside at least one of the polyhedra; 

if a volume part is at least partially within alt of the polyhedra, processing the data defining the planar 
surfaces which intersect the volume part and, In dependence upon the number of planar surfaces which 
intersect the volume part: 

subdividing the volume part into a plurality of smaller volume parts for further testing, 

calculating the 3D points in the volume part at which a predetennined number of the planar surfaces meet; or 
discarding the volume part. 



6. A method according to claim 5, wherein a volume part is subdivided if the volume part is at least partially 
within all of the polyhedra and the number of planar surfaces which intersect the volume part is more than a 
predetermined number. 



7. A method according to claim 5 or claim 6, wherein a volume part is discarded if the volume part is at least partially 
within all of the polyhedra and the number of planar surfaces which intersect the volume part is less than a 
predetermined number. 

8. A method according to any of claims 5 to 7, wherein, when a volume part is tested, data is stored defining the 
planar surfaces which Intersect the volume part, and when a smaller volume part is tested which vsas generated by 
subdivision of a parent volume part, only the planar surfaces which intersect the parent volume part are tested 

to to determine whether they intersect the smaller volume part. 

9. A method according to any of claims 5 to 8, wherein, when a volume part is tested, data is stored defining each 
polyhedron which the volume part is determined to lie wholly inside, and, when a smaller volume part is tested 
which was generated by subdivision of a parent volume part, tests are carried out to determine whether the 
smaller volume part lies inside each of the polyhedra not defined in the stored data, but no tests are carried 

^5 out to determine whether the smaller volume part lies inside any of the polyhedra defined in the stored data. 

10. A method according to any of claims 3 to 9, wherein a volume part is tested against a polyhedron to determine 
if it lies wholly outside the polyhedron by determining whether any of the polygon faces of the polyhedron 
intersect the volume part and: 

20 

in the event that at- least one polygon face intersects the volume part, detemiining that the volume part 
lies at least partially within the polyhedron; 

in the event that none of the polygon faces intersect the volume part, projecting a ray from a point within 
the volume part to a predetermined point in the three-dimensional space, and determining whether the volume 
25 part lies wholly inside the polyhedron or wholly outside the polyhedron in dependence upon the number of 

intersections of the ray with the polyhedron. 



11. A method according to any preceding claim, wherein each respective polyhedron comprises a projection into the 
three-dimensional space of the outline of the subject object from a two-dimensional image having a known position 
and orientation in the three-dimensional space, each respective polyhedron thereby representing the shape of the 
subject object in a plane parallel to the plane of the image from which it was generated. 



12. A method according to claim 1 1 , further comprising the steps of: 



35 processing data defining a plurality of images of the subject object to approximate the outline of the . 

subject object in each image with straight lines; 

projecting the straight lines into the three-dimensional space in dependence upon the positions and 
orientations of the images in the three-dimensional space, to define the polyhedra, each respective 
polyhedron representing the shape of the subject object in a plane parallel to the plane of the image from 
^0 which it was generated. 



13. A method according to claim 11 or claim 12, wherein the step of testing a volume part against a polyhedron 
comprises projecting the volume part into the two-dimensional image corresponding to the polyhedron and testing 
the two-dimensional projected volume part against the outline of the subject object in the image. 

14. A method according to claim 13, wherein the outline of the subject object in each two-dimensional image 
comprises a plurality of straight line edges, and wherein, to determine the planar surfaces which intersect the 
volume part, the two-dimensional projected volume part is tested to determine the outline edges that intersect the 
projected volume part. 

50 

15. A method according to claim 14, wherein the straight line edges of the outline in each two-dimensional image 
are connected end-to-end at vertices, and wherein, to determine the planar surfaces which intersect the volume 
part, the two-dimensional projected volume part is tested to detemnine the outline vertices which lie therein and 
the outline edges that intersect the projected volume part. 



55 



16- A method according to claim 14 or claim 15, wherein, to determine the outline edges vs^iich intersect a projected 



volume part: 

a portion of the image containing the projected volume part is identified; 

stored data defining regions of the image and the outline edges within each region is read; and 

the projected volume part is tested only against outline edges defined in the stored data to lie in regions 
which overlap the identified portion of the image containing the projected volume part. 



17- A method according to any of claims 1 to 10, wherein each respective polyhedron comprises a polyhedron 
having a hole therein, the hole defining the outline three-dimensional shape of a part of the subject object. 

18. A method of processing data defining a plurality of images of a subject object to generate data defining a 3D 
computer model of the subject object, comprising: 

processing each image to project the outline of the subject object therein into three-dimensional space in 
dependence upon the position and orientation of the image in the three-dimensional space, to generate a 
respective polyhedron from each image comprising a plurality of planar faces defining the visual hull of the 
subject object; and 

calculating the points in the three-dimensional space at which a predetermined number of the planar faces 
from at least two respective polyhedra intersect by: 

defining a volume of the three-dimensional space in which all points, on the surface of the subject 
object must lie; 

subdividing the volume into a plurality of volume parts and storing data defining a list of the volume parts; 
selecting a volume part from the list, testing it against the polyhedra generated from the images, and: 
if the volume part It is wholly outside at least one of the polyhedra, discarding the volume part; 

- if the volume part is at least partially within all of the polyhedra: TOO MANY NESTINGS IN USTS, 
LEVEL4gSMiXED 

subdividing the volume part into smaller volume parts and adding each smaller volume part to the list 
of parts to be processed if the planar faces which intersect the volume part are sufficient in number 
to intersect to generate more than a predetemiined number of 3D points representing points on the 
surface of the subject object; 

- discarding the volume part if the planar faces which intersect the volume part are not sufficient in 
number to intersect to generate at least one 3D point representing a point on the surface of the 
subject object; 

- calculating 3D points in the volume part representing points on the surface of the subject object if 
the number of planar faces intersecting the volume part is equal to the number required to intersect 
to generate a predetemiined number of 3D points representing points on the surface of the subject 
object; 



repeating the step of selecting a volume part from the list and testing it until all volume parts on the list have 
been selected and tested, thereby to generate a 3D computer model of the subject object comprising points in 
the three-dimensional space defined by the intersection of planar faces from the polyhedra. each 3D point 
representing a point on the visual hull of the subject object. 



A method according to any of claims 4 to 18, further comprising the step of connecting the calculated points in 
the three-dimensional space representing points on the surface of the subject object to generate a 3D computer 
model compnsing a polygon mesh representing the surface of the subject object. 

A method according to any preceding claim, further comprising the step of generating a signal conveying the 3D 



computer model. 



A method according to claim 20, further comprising the step of making a recording of the signal either directly 

or indirectly. 

Apparatus for processing data defining at least three polyhedra, each respective polyhedron comprising a 
plurality of planar faces in a three-dimensional space defining a volume within which at least part of a subject 
object lies, to generate a 3D computer model of the subject object comprising points in the three-dimensional 
space representing points on the surface of the subject object connected to form a polygon mesh, the apparatus 
comprising: 

3D point calculating^ means for calculating points in the three-dimensional space representing points on the 
surface of the subject object by calculating the respective points at which a predetermined number of planar 
faces from the polyhedra intersect; 

3D point identification means for assigning an identification to each calculated point representing a point on 
the subject object in dependence upon the planar faces which meet at the point; and 

3D point connecting means for connecting the points in dependence upon the assigned identifications to generate 
a 3D computer model comprising a polygon mesh representing the surface of the subject object, the 3D point 
connecting means being arranged to connect the points after processing by the 3D point calculating means to 
calculate all points in the 3D computer model is complete. 



Apparatus according to claim 22, wherein the 3D point calculating means is arranged to carry out tests to 
identify which planar faces cannot intersect at a point on the subject object by testing volumes of the three- 
dimensionar space to identify volumes which do not contain any part of the subject object and the planar faces 
intersecting the identified volumes, and wherein the 3D point calculating means is arranged not to calculate 
points of intersection for the identified planar faces which cannot intersect at a point on the subject object. 

Apparatus according to claim 23, wherein the 3D point calculating means comprises: 

volume defining means for defining a volume of the three-dimensional space in which all points on the subject 

object willlie; 

volume dividing mearis for dividing the volume into a plurality of volume parts; 
volume testing means for testing each volume part against the polyhedra, and for: 
discarding the volume part if it is wholly outside at least one of the polyhedra; 

if the volume part is at least partially within all of the polyhedra, processing the data defining the 
planar surfaces which intersect the volume part to calculate the 3D points in the volume part at which a 
predetermined number of the planar surfaces meet. 



Apparatus for processing data defining a plurality of polyhedra, each respective polyhedron comprising a 
plurality of planar surfaces in a three-dimensional space defining a volume within which at least part of a 
subject object lies, to generate a 3D computer model of the subject object comprising points in the three- 
dimensional space representing points on the surface of the subject object, the apparatus comprising: 

volume defining means for defining a volume of the three-dimensional space in which all points in the 3D 
computer model vwll lie; 

volume dividing means for dividing the volume into a plurality of volume parts; 

volume testing means for testing each volume part against the polyhedra, the volume testing means being arranged 
to: 

discard the volume part If it is v\lYolly outside at least one of the polyhedra; 

if the volume part is at least partially within all of the polyhedra, process the data defining the planar 
surfaces which intersect the volume part to calculate the 3D points in the volume part at which a predetermined 
number of the planar surfaces meet. 



Apparatus according to claim 24 or claim 25, wherein the volume dividing means and the volume testing means are 
arranged to perform processing to: 

discard a volume part if it is vvh oily outside at least one of the polyhedra; 

if a volume part is at least partially within all of the polyhedra, process the data defining the planar surfaces 
which intersect the volume part and, in dependence upon the number of planar surfaces which Intersect the volume 
part: 

subdivide the volume part into a plurality of smaller volume parts for further testing, 

calculate the 3D points in the volume part at which a predetermined number of the planar surfaces meet; or 

discard the volume part. 



Apparatus according to claim 26, wherein the volume dividing means and the volume testing means are arranged to 
subdivide a volume part if the volume part is at least partially within all of the polyhedra and the number of 
planar surfaces which Intersect the volume part is more than a predetermined number. 

Apparatus according to claim 26 or claim 27, wherein the volume dividing means and the volume testing means are 
arranged to discard a volume part if the volume part Is at least partially within all of the polyhedra and the 
number of planar surfaces which intersect the volume part is less than a predetermined number. 

Apparatus according to any of claims 26 to 28, wherein the volume testing means is arranged to store data when a 
volume part is tested defining the planar surfaces which intersect the volume part, and is arranged to operate so 
that, when a smaller volume part is tested which v\as generated by subdivision of a parent volume part, only the 
planar surfaces which Intersect the parent volume part are tested to determine whether they intersect the smaller 
volume part. 

Apparatus according to any of claims 26 to 29, wherein the volume testing means is arranged to store data when a 
volume part is tested defining each polyhedron which the volume part is determined to lie wholly inside, and Is 
arranged to operate so that, when a smaller volume part is tested which was generated by subdivision of a parent 
volume part, tests are carried out to determine whether the smaller volume part lies inside each of the polyhedra 
not defined in the stored data, but no tests are carried out to determine whether the smaller volume part lies 
Inside any of the polyhedra defined in the stored data. 

Apparatus according to any of claims 24 to 30, wherein the volume testing means is arranged to test a volume part 
against a polyhedron to determine if it lies wholly outside the polyhedron by determining whether any of the 
polygon faces of the polyhedron intersect the volume part and: 

in the event that at least one polygon face intersects the volume part, determine that the volume part lies at 
least partially within the polyhedron; 

in the event that none of the polygon faces intersect the volume part, project a ray from a point within the 
volume part to a predetermined point in the three-dimensional space, and determine whether the volume part lies 
wholly inside the polyhedron or wholly outside the polyhedron in dependence upon the number of intersections of 
the ray with the polyhedron. 



Apparatus according to any of claims 22 to 31 , wherein each respective polyhedron comprises a projection into the 
three-dimensional space of the outline of the subject object from a two-dimensional image having a known position 
and orientation in the three-dimensional space, each respective polyhedron thereby representing the shape of the 
subject object in a plane parallel to the plane of the image from which it was generated. 

Apparatus according to claim 32, further comprising: 

outline approximation means for processing data defining a plurality of images of the subject object to 
approximate the outline of the subject object in each image with straight lines; and 

projection means for projecting the straight lines into the three-dimensional space in dependence upon the 



positions and orientations of the images in the three-dimensional space, to define the polyhedra, each respective 
polyhedron representing the shape of the subject object in a plane parallel to the plane of the image from which 
it was generated. 



Apparatus according to claim 32 or claim 33, wherein the volume testing means is arranged to test a volume part 
against a polyhedron by projecting the volume part into the two-dimensional image corresponding to the polyhedron 
and testing the two-dimensional projected volume part against the outline of the subject object in the image. 

Apparatus according to claim 34, wherein the outline of the subject object in each two-dimensional image 
comprises a plurality of straight line edges, and wherein, to determine the planar surfaces which Intersect the 
volume part, the volume testing means is arranged to test the two-dimensional projected volume part to determine 
the outline edges that Intersect the projected volume part. 

Apparatus according to claim 35, wherein the straight line edges of the outline in each two-dimensional image are 
connected end-to-end at vertices, and wherein, to determine the planar surfaces which intersect the volume part, 
the volume testing means is arranged to test the two-dimensional projected volume part to determine the outline 
vertices which lie therein and the outline edges that intersect the projected volume part. 

Apparatus according to claim 35 or claim 36, wherein the volume testing means is arranged to perform processing 
to determine the outline edges which intersect a projected volume part by: 

Identifying a portion of the image containing the projected volume part; 

reading stored data defining regions of the image and the outline edges within each region; and 

testing the projected volume part only against outline edges defined in the stored data to lie in regions which 
overlap the identified portion of the Image containing the projected volume part. 



Apparatus according to any of claims 22 to 31, wherein each respective polyhedron comprises a polyhedron having 
a hole therein, the hole defining the outline three-dimensional shape of a part of the subject object. 

Apparatus for processing data defining a plurality of images of a subject object to generate data defining a 3D 
computer model of the subject object, comprising: 

means for processing each Image to project the outline of the subject object therein into three-dimensional 
space in dependence upon the position and orientation of the Image In the three-dimensional space, to generate a 
respective polyhedron from each Image comprising a plurality of planar faces defining the visual hull of the 
subject object; and 

means for calculating the points in the three-dimensional space at which a predetermined number of the planar 
faces from at least two respective polyhedra intersect by: 

defining a volume of the three-dimensional space in which all points on the surface of the subject object 
must lie; 

subdividing the volume Into a plurality of volume parts and storing data defining a list of the volume parts; 
selecting a volume part from the list, testing It against the polyhedra generated from the images, and: 

if the volume part It Is wholly outside at least one of the polyhedra, discarding the volume part; 

if the volume part Is at least partially within all of the polyhedra: 

subdividing the volume part into smaller volume parts and adding each smaller volume part to the list 
of parts to be processed if the planar faces which intersect the volume part are sufficient in number 
to intersect to generate more than a predetermined number of 3D points representing points on the 
surface of the subject object; 

discarding the volume part if the planar faces which intersect the volume part are not sufficient in 
number to intersect to generate at least one 3D point representing a point on the surface of the 
subject object; 
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TO THE 3D POINT COMPRISING A 
TRIPLE FORMED FROM THE 
THREE IDs OF THE PLANES 
WHICH MEET AT THE 3D POINT 



S25-12 




YES^ ANOTHER 

CALCULATED 3D POINT? 




S25-14 



NO 



Fig.26. 

S23-4: ORGANISE 
THE CALCULATED 3D 
POINTS INTO SETS IN 
DEPENDENCE UPON THE 
SIGNATURES 



CONSIDER NEXT CALCULATED 
3D POINT 



READ THE SIGNATURE OF THE 
3D POINT 



ALLOCATE THE 3D POINT TO THE 
SET OF POINTS FOR EACH 
PLANE DEFINED IN THE 
SIGNATURE 



S26-2 



S26-4 



S26-6 



,^_)II|^-^NOTHER CALCUUVTED 
3D POINT? 



C 




S26-8 



NO 



RETURN 



Fig.29. 

1110 



1100 



NO 



Fig.30a. 



S23-6: DETERMfNE THE 
(CONNECTION ORDER OF THE 3D 
POINTS IN EACH SET 



CREATE AN EMPTY POLYGON 
SET "S" FOR THE 3D COMPUTER 
MODEL 



CONSIDER THE NEXT PU\NE "p" 
AND READ THE 3D POINTS IN 
THE SET Vp FOR THE PLANE 




S30-2 

FROM 
-<n STEP 
^ S30-56 



S30-4 



ARE THERE ANY 3D 
POINTS IN THE SET Vp 



YES 




S30-6 



ADO A NEW POLYGON DATA 
STRUCTURE "s" TO THE 
POLYGON SET "S" AND STORE 
THE NORMAL VECTOR FOR THE 
POLYGON IN THE DATA 
STRUCTURE 



SELECT A 3D POINT FROM THE 
SET Vp AS A START VERTEX u 
FOR THE POLYGON 



SET CURRENT VERTEX POINTER 
WTO VERTEX u 



3 



READ SIGNATURE OF START 
VERTEX u 



REMOVE THE CURRENT 
VERTEX w FROM THE SET Vp 
FOR PLANE p 



ytLECT A PUVNb "q". DEFINED IN 
THE SIGNATURE (q NOT EQUAL 
TOP) 



S30-18 



© 



© Fig. 30b. 



S30-20 



ADD I HE VERTEX TO THE 
POLYGON DATA STRUCTURE "s" 
TO DEFINE A VERTEX OF THE 
POLYGON "s" 



S30-22 



DETERMINE THE SET Vpq OF 3D 
POINTS WHICH REMAIN IN THE 
SET Vp AND HAVE "q" IN THEIR 
SIGNATURE 



S30-24 




ARE THERE ANY 3D 
POINTS IN THE SET Vpq? 




NO 



YES 



IS SIZE (Vpq) = 1 




NO. 



SET NEXT VERTEX 
"x" OF THE 
POLYGON "s" TO 
BE THE 3D POINT 
IN 

THE SET Vpq 



S30-28 



S30-32 



I 



DEFINE A SET, Vline, OF 3D 
POINTS LYING ON THE 
INTERSECTION OF PLANES p 
AND q CONTAINING THE 3D 
POINTS w AND u AND THE 3D 
POINTS IN THE SET Vpq 



NO 




ARE THERE 
AN ODD NUMBER 
OF 3D POINTS IN 
THE SET Vpq? 




S30-30 



S30-34 



DEFINE A SET, Vline. OF 3D 
POINTS LYING ON THE 
INTERSECTION OF PLANES p 
AND q CONTAINING THE 3D 
POINT w AND THE 30 
POINTS IN THE SET Vpq 



© 



Fig.SOc. 



RANK tACH 30 POINT IN THE 
SET VIme BY RELATIVE POSITION 
ALONG THE INTERSECTION OF 
PLANES p AND q, STARTING 
FROM RANK 0 



S30-36 



S30-38 




IS RANK OF CURRENT 
VERTEX w EVEN? 

S30-40 



SET NEXT VERTEX V OF 
THE POLYGON "s" TO BE 
THE VERTEX "v" IN THE SET 
Vpq SUCH THAT: 
rank (v) = rank (w) h i 




S30-42 



SET NEXT VERTEX "x" OF 
THE POLYGON "s" TO BE 
VERTEX V IN THE SET Vpq 
SUCH THAT: 
rank (v) = rank (w) -i 



J 



SET CURRENT VERTEX 
POINTER w TO VERTEX x 



READ THE SIGNATURE OF 
THE CURRENT VERTEX 



SELECT THE PLANE "r" 
DEFINED IN THE SIGNATURE 
(r NOT EQUAL TO p AND r 
NOT EQUAL TO q) 




S30-44 



S30-46 



-S30-48 



Fig.SOd 



NO 



FROM STEP S30-6 




S30-50 



IS THERE MORE 
THAN ONE POLYGON IN 
THE PLANE? 




YES 



TEST EACH POLYGON TO 
DETERMINE WHETHER IT 
CONTAINS ANY OF THE 
OTHER POLYGONS 



"S30-52 



DETERMINE WHETHER 
EACH POLYGON 
REPRESENTS SURFACE OF 
THE SUBJECT OBJECT OR A 
HOLE THEREIN 




^330-54 



ANOTHER PU^NE? 



S30-56 



YES 



c 





TO STEP 
S30-4 



NO 



RETURN 



Fig.32. 

START ^ 



GENERATE 3D COMPUTER 
MODEL OF EACH COMPONENT 
OBJECT AND DISPLAY IMAGE{S) 



S32-2 



POSITION COMPONENT 
OBJECTS IN A COMMON 
COORDINATE SPACE AS PARTS 

OF A REQUIRED COMPOUND 
OBJECT AND DISPLAY IMAGE(S) 
IN ACCORDANCE WITH USER 
INPUT INSTRUCTIONS 



S32-4 



CALCULATE 3D POINTS 
DEFINING VERTICES OF THE 
COMPOUND OBJECT 



S32-6 



CONNECT CALCULATED 3D 
POINTS TO GENERATE 
POLYGON MESH 
REPRESENTING THE 

SURFACE OF THE 
COMPOUND OBJECT 



S32-8 



OUTPUT DATA AND/OR DISPLAY 
IMAGE(S) OF THE COMPUTER 
MODEL OF THE COMPOUND 
OBJECT 



S32-10 




Fig.33a. 
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2240 




2200 



2230 



2260 



2290 




2330 
/ 



2210 



2310 > 

2300 




2350 



2400 / 2380 
2370 



Fig.33b. 




/S32-6; CALCULA re 3D POIN l9\ 
( DEFINING VERTICES OF THE ) 
\^ COMPOUND OBJECT J 



Fig. 34, 



DEFINE A 3D VOLUME IN WHICH ALL 

CALCULATED POINTS ON THE 
COMPOUND OBJECT SURFACE WILL 
LIE 



SUBDIVIDE THE VOLUME AND ADD 
THE CHILD VOLUMES TO THE TOP 
OF A S l ACK 



DEFINE A RFSPECTIVE 
POLYHEDRON FOR EACH 
COMPONENT OBJECT CONTAINING 

THE COMPONENT OBE JCT AS A 
HOLE 



S34-2 



S34-4 



S3 4-6 



POP NEXT VOLUME OFF THE TOP 
OF THE STACK 



S3 4-8 



DETERMINE THE STATE OF THE 
VOLUME 



^S34-10 




IS THE STATE OF 
THE VOLUME 
^CALCULATE VERTEX"? 



5 




S34-12 



YES 



CALCULATE VERTEX 



5 



IS THE STATE OF THE 
VOLUME "DISCARD" OR 
"SUBDIVIDE"? 




^ DISCARD 



SUBDIVIDE 
VOLUME AND 

ADD CHILD 
VOLUMES TO 
THE TOP OF THE 
STACK 



834-18 



DISCARD THE 
VOLUME 



Z 



ANOTHER VOLUME 
ON THE S I ACK? 



C 



^ HO 




RETURN 



3 



Fig.35. 




Fig.36. 




Fig.37 




Fig.38a. 



(! 



S34-10: DETERMINE THE STATE Of' 
THE V OLUME 



CONSIDER THE LIST OF POLYGONS 
WHICH INTERSECT THE PARENT 
VOLUME 



-.S38-2 



CONSIDER THE NEXT POLYGON ON 
THE LIST 



S38-4 



NO 



DOES THE 
POLYGON INTERSECT THE 
CURRENT VOLUME-? 




S38-6 



YES 



ADD THE POLYGON TO THE LIST OF 
POLYGONS WHICH INTERSECT THE 
CURRENT VOLUME 




S38-8 



ANOTHER POLYGON 
ON THE LIST 




S38-10 
YES 



NO 



COPY THE LIST OF ACTIVE 
POLYHEDRA FOR THE PARENT 
VOLUME AS A LIST FOR THE 
CURRENT VOLUME 



.S38-12 



NO 



ARE THERE ANY ^ 
POLYHEDRA ON THE LIST 
HAVING NO POLYGONS 
WHICH INTERSECT THE 
CURRENT VOLUME? 



S38-14 



YES 



Fig.38b. 



CONSIDER THE NEXT POLYHEDRON 
ON THE LIST HAVING NO POLYGONS 
WHICH INTERSECT THE CURRENT 
VOLUME 



'S38-16 



TEST WHETHER THE 
POLYHEDRON ENCLOSES OR IS 
DISJOINT FROM THE CURRENT 
VOLUME 



S38-18 




IS THE CURRENT 
VOLUME ENCLOSED BY OR 
DISJOINT FROM THE 
POLYHEDRON 




S38-20 



DtSJOINT 



ENCLOSED S38-24 



DELETE Tl IE POLYHEDRON FROM 
THE LIST OF ACTIVE POLYHEORA 
FOR THE CURRENT VOLUME 



YES 




S38-26 



ANOTHER 
POLYHEDRON 
ON THE LIST HAVING NO 
POLYGONS WHICH 

INTERSECT THE 
CURRENT VOLUME? 

I NO 




V S38-22 



1 



SET STATE OF 
CURRENT 

VOLUME TO 
"DISCARD" 



RETURN 



Fig.38c. 

— L__ 

CONSIDER THE LIST OF POLYGONS 
WHICH INTERSECT THE CURRENT 
VOLUME 



<3 



S38-32 



IS THE NUMBER OF , 
POLYGONS ON THE LIST 3 
MORE THAN 3, OR LESS 
THAN 3? 



1 



SET STATE OF 
VOLUME TO 
"DISCARD" 



I 



SET STATE OF 
VOLUME TO 
"CALCULATE 
VERTEX" 




S38-36 



SET STATE OF 
VOLUME TO 
"SUBDIVIDE" 



RETURN 



RETURN 



3 C 



RETURN 



3 



Fig.39. 



538-18: TEST WHETHER THE 
POLYHEDRON ENCLOSES OR 
IS DISJOINT FROM THE 
CURRENT VOLUME 



SELECT A POINT ON THE SURFACE 
OF THE VOLUME 



S39-2 



PROJECT A RAY FROM THE 
SELECTED POINT TO 
PREDETERMINED DISTANCE AND 
COUNT THE NUMBER OF 
INTERSECTIONS OF THE RAY WITH 
THE POLYHEDRON 



S39-4 



ODD 



S39-8 



IS THE NUMBER OF 
'INTERSECTIONS OF THE RAY^ 
sWITH THE POLYHEDRON ODD^ 
OR EVEN? 



S39-6 



EVEN 



i 



SET 
"ENCLOSED" 
FLAG 



S39-10 



SET "DISJOrNT" 
FLAG 



c 



RETURN 



3 



Fig.40a. 




Fig. 41 a. 




Fig.40b. 




2600 



Fig.41b. 




Fig.42. 



NO 



S34-14: CALCULATE A 
VERTEX J 



CALCULATE THE 3D 
POSITION OF THE POINT 
AT WHICH THE PLANES 

CONTAINING THE 
POLYGONS INTERSECT 



YES 



ADD THE CALCUU^TED 3D 
POINT TO THE LIST OF 3D 
POINTS REPRESENTING 
VERTICES OF THE 
COMPOUND OBJECT 



C 



RETURN 



-S42-2 



s^^ S42-4 

'IS THE CALCULATED\y^ 
3D POINT INSIDE THE 3D 
^ VOLUME AND WITHIN ALL / 
3 POLYGONS? 



S42-6 



-> 


r 




SET STATE OF VOLUME 
TO "DISCARD" 



S42-8 



Fig.43. 




Fig.44. 



S23-2: GENERATE 
/PLANE DATA AND DETERMINE ' 
THE "SIGNATURE" OF EACH 
CALCULATED 3D POINT 



CONSIDER THE NEXT 
POLYGON USED TO 
CALCULATE A VERTEX OF THE 
COMPOUND OBJECT 



S44-2 



ASSIGN A UNIQUE ID TO THE 
POLYGON UNLESS AN ID HAS 
ALREADY BEEN ASSIGNED TO 
THE POLYGON 



YES 




S44-4 



844-6 



ANOTHER POLYGON 
USED TO CALCULATE A 
VERTEX OF THE 
OMPOUND OBJECT?, 




NO 



CONSIDER THE NEXT 
CALCULATED VERTEX OF THE 
COMPOUND OBJECT 



S44-8 



ASSIGN A SIGNATURE TO THE 
VERTEX COMPRISING A 
TRIPLE FORMED FROM THE 

THREE IDS OF THE 
POLYGONS WHICH MEET AT 
THE VERTEX 



Y£S 




S44-10 



S44-12 



ANOTHER 
CALCULATED VERTEX 0f\ 
THE COMPOUND 

OBJECT? >^ 



C 



NO 



RETURN 



1 



Fig.45a. 



S4- 1 6: GENERATE ^\ 
/ DATA DEFINING A 3D \ 
I SURFACE MODEL OF THE / 
\^ SUBJECT OBJECT ~/ 



APPROXIMATE THE 
SILHOUETTE OF THE SUBJECT 
OBJECT IN EACH SEGMENTED 
IMAGE WITH STRAIGHT LINES 



S45-2 



PROJECT EACH STRAIGHT 
LINE OF EACH SILHOUETTE 

INTO 3D TO GIVE A 
RESPECTIVE POLYHEDRON 
COMPRISING PLANAR FACES 
FOR EACH IMAGE. ASSIGN A 
UNIQUE ID TO EACH PLANAR 
FACE 



CONSIDER THE NEXT SET OF 

THREE PLANAR FACES 
CONTAINING FACES FROM AT 
LEAST TWO POLYHEDRA 



S45-4 



S45-6 



CALCULATE THE POINT OF 
INTERSECTION OF THE 
THREE PLANAR FACES 



S45-8 



Fig. 45b. 



IS THE POINT OF 
INTERSECTION WITHIN 
%v ALL OF THE 

POLYHEDRA? 




YES 



IS THE POINT OF 
INTERSECTION WITHIN 
ALL OF THE THREE 
PLANAR FACES? 




S45-10 



S45-14 



YES 



RETAIN THE POINT AND ASSIGN 
A SIGNATURE TO IT COMPRISING 
A TRIPLE FORMED FROM THE 
THREE IDs OF THE PLANAR 
FACES WHICH MEET AT THE 
POINT 



S45-16 



S45-12 



DISCARD 
THE POINT 



YES 



ANOTHER SET OF 
THREE PLANAR FACES 
CONTAINING FACES FROM \ 
AT LEAST TWO 
POLYHEDRA? 



=ROM ^ 



S45-18 



NO 



CONNECT THE CALCULATED 
3D POINTS TO GENERATE 
SURFACE POLYGONS 



S45-20 



C 



RETURN 



0 



'^ im PAGE BLAT^I^ (uspto) 



< 



